Artificial Intelligence PTF

These notes are also filed as OMG document ai/20-01-01.

This two-hour event, taking place during “happy hour” (5-7 pm) on Monday evening, was open to the public and attracted an audience of about 70. The short program, leaving plenty of time for networking and enjoying the food and bar, was in four parts:

• Terry McElrath (OMG) introduced the event
• Claude Baudoin explained the history and scope of OMG’s work on AI standards
• Elisa Kendall (Thematix Partners) presented OMG’s work on knowledge representation, mostly done under the Ontology Platform SIG
• A lively panel, moderated by Claude, featured Elisa, Roger Burkhart (John Deere), and Artem Beer (Torch.AI).

Claude Baudoin (cébé IT & Knowledge Management) and Bobbin Teegarden co-chair the AI PTF. Claude Baudoin led the meeting, held on Wednesday afternoon, which attracted over 20 participants.

Claude gave a summary of the past events leading to this meeting:

• In March, the OMG Board asked for the development of an AI standards strategy.
• On May 1, NIST issued an RFI on AI Standards. The strategy paper was set aside in order to devote time to a response, sent to NIST on June 6. NIST received 98 responses in total.
• The strategy paper development resumed in July and August, with help from Pete Rivett, Elisa Kendall, Sridhar Iyengar, and a few others. The paper benefited from the work done on the NIST RFI, and was completed and sent to the Board on August 26.
• Meanwhile, at the June meeting in Amsterdam, the Agent PSIG was “morphed” into an AI PSIG. However, it was the opinion of OMG management that a Task Force would be more visible, and they pushed for this to be proposed at the September meeting in Nashville.
• A “formation meeting” was held on September 25 and followed by a workshop on Augmented Reality. An AI PTF charter, published earlier, was approved by the PTC on September 27. A report from this meeting was sent to the [email protected] mailing list a week before this meeting, and is filed as OMG document ai/19-12-01.

Upon review of the charter, three changes were proposed and approved by the PTF, resulting in document ai/19-12-03. They were subsequently adopted during the PTC plenary session on Friday morning. Both changes related to bullets in the scope statement within the charter:

• The third bullet, “image and sound recognition,” becomes “Image understanding, speech recognition, and computer vision”
• A new fourth bullet, “Robotic Systems,” was added
• The fifth (now sixth) bullet, “autonomous systems and agents,” becomes “autonomous and autonomic systems and agents.”

The most common question we have heard -– and tried to answer -– so far is, “what is worth and useful to standardize in AI?” Behind that question lurks a suspicion that there are some standardization efforts that might have a stifling effect on innovation, and that we need to avoid this undesirable effect. The following ideas were collected (some topics were restructured for consistency, and footnotes were added for clarification):

• A “notional AI architecture”

  • See Ed Feigenbaum’s three books on AI – presumably referring to Computers and Thought, the Handbook of Artificial Intelligence, and The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to the World.
  • See what university courses describe.
  • See the AAAI’s taxonomy of areas of AI – this appears in an Appendix in the OMG AI Strategy paper. It is not an architecture, just a taxonomy, and it only appears in the form of a menu of search options within the AITopics publication, under the “Technology” facet (left column).
  • It needs to document the various “paradigms of AI.”
• Integrity of systems

  • Auditability and bias (e.g., avoid bias in lending decisions, in social media filters, in medical advice engines…)
  • Ability to check the consistency of decisions
  • Transparency – this would include the “explainability” of machine learning algorithms
  • What kinds of AI are being used?
  • Define clear requirements for bias avoidance, e.g. in terms of percentages of false positives or false negatives
• APIs

  • What do AI applications need from AI platforms? What APIs would make those AI applications portable across platforms?
• Find who is working on what in other OMG subgroups as well as outside

  • Some initiatives are listed in the AI strategy paper
  • See what else is suggested in the 97 other responses to the NIST RFI on AI Standards
• Modeling of AI Components
• Extend the Architecture Modernization Task Force’s Abstract Syntax Tree Metamodel (ASTM) to AI languages
• Also extend CISQ’s metrics (adopted as OMG standards through the same ADM Task Force) to AI languages.
• Guidelines on where the biggest RoI for AI is
• A glossary of all the terms we have been using

  • There is a Glossary of Artificial Intelligence in Wikipedia, don’t redo this work if it is satisfactory
  • See the disciplines and courses covered at MIT’s CSAIL (Computer Science and AI Lab)
• Something to allow the automatic security classification of data

  • See Mike Abramson’s work on IEF/DCS (next section below)

Mike Abramson (ASMG) had asked to present a summary of the status of his work on IEF/DCS and the relevance of AI to it. His short slide set is document ai/19-12-02. IEF provides an architecture to manage the restrictions on interchange of confidential documents and e-mails, but the work of classifying an enterprise’s information assets in order to apply IEF policies to them is impossible to perform manually. Therefore, some automatic and intelligent classification process is going to be necessary.

This second brainstorm aimed at populating our roadmap for 2020 and beyond.

As emphasized during the September meeting, the AI PTF needs to work closely with other OMG subgroups to (a) collect their requirements, (b) offer our help, © avoid duplicating their efforts or taking over what they may be already doing very well – especially, but not only, in the Ontology PSIG or the Robotics DTF. To this effect, we listed a number of target groups and their putative “correspondents” (some of them, identified with a question mark, were assigned in absentia) to ensure this coordination. The responsibilities we wish to assign to these correspondents are:

• Help us get time on their own agendas to present what we are doing
• Provide the AI PTF with their groups’ priorities and requirements regarding AI
• Convey to their members any proposals we wish to make, so we can receive direct feedback.

The list of proposed liaisons is as follows, with question marks for people who were not in the room and whose agreement has been assumed rather than verified.

This was a second brainstorm about activities we need to undertake –- not quite a roadmap yet, but aimed to lead us toward creating one.

• Liaise with other OMG subgroups

  • See the previous subsection
• Complete the review of 98 responses to NIST’s RFI on AI Standards (97 excluding OMG's):

  • Claude showed the existing Google Sheet, partially filled by Pete Rivett and him.
  • We need more volunteers to (a) analyze the remaining responses, (b) review the existing analyses and provide additional opinions if desired.
• Add new content to our wiki

  • List of upcoming conferences
  • More references
  • The current page was created and maintained by Claude – we need more co-authors!
• Create a “reference model” or “landscape” of AI

  • We would love for NIST to do something similar to their authoritative “cloud computing reference model” of 2011. Allison Barnard-Feeney said that NIST is receiving a significant amount of money for its AI program, and is hiring right now to staff this.
  • If we do something ourselves, we need to resolve the language -– is it a reference architecture, reference model, a landscape? Mike Bennett said that one size does not fit all. Claude described the shape of it as a “bento box.”
• Query universities and research centers on their current research projects

  • Find what’s new and exciting in AI
  • Stanford, MIT CSAIL, MIT Media Lab, Frankfurt (Roberto Zicari’s work on inspection of AI), SINTEF in Norway, and many more…
• Potentially issue a complementary RFI to NIST's

  • to ask additional questions
  • to involve non-US entities that might have been discouraged by NIST's questions aimed at ensuring or reinforcing the competitiveness of American companies.
• Transform our findings into discussion papers.

At a later point in the meeting, we transformed this into a clearer roadmap for 2020:

There was a consensus that we should strive for a three-part agenda in Reston:

• A regular half-day meeting of the Task Force, focused on our internal status review, planning, work on an RFP, etc.
• Joint sessions with other Task Forces and SIGS – for which we need to create a standard slide deck that any of us can take to those bodies and deliver a consistent message
• A full-day “AI Forum” with papers and a panel:

  • on a day selected in coordination with OMG’s MarCom team, since space is at a premium during our meetings in Reston,
  • preceded by a Call for Papers that would specifically solicit presentations on AI landscapes, architectures, reference models, ontologies, etc.,
  • heavily marketed to our contacts within NIST, the DoD, and other Washington area stakeholders.

Philip Newcomb volunteered to form and lead a Program Committee for this event. In order to pull this off, we will need to have several regular conference calls between now and then. There are several topics that could be presented/discussed either at the smaller internal meeting or at the forum:

• a presentation by Torch.AI,
• a discussion with the Joint AI Center (JAIC) facilitated through Steve MacLaird,
• a repeat presentation by Andreas Vogel on AI Ethics.

In closing, we briefly recapitulated the action items that need to be executed in the near future. Lars Toomre suggested that we exchange the e-mail addresses of everyone who attended this meeting, so we know each other.

Bobbin asked how we should collect a list of conferences and activities to follow. Claude agreed to receive such messages and to add the suggestions to the wiki.

Given the need to coordinate the March schedule but also to act on OMG management's urgent desire to see a first RFP issued soon, we need to schedule teleconferences in January and February. Claude will “doodle” this.