Notes from AI Meet-and-Greet and AI PTF Meeting, December 2019, Long Beach

These notes are also filed as OMG document ai/20-01-01.

AI Meet-and-Greet

This two-hour event, taking place during “happy hour” (5-7 pm) on Monday evening, was open to the public and attracted an audience of about 70. The short program, leaving plenty of time for networking and enjoying the food and bar, was in four parts:

Terry McElrath (OMG) introduced the eventClaude Baudoin explained the history and scope of OMG’s work on AI standardsElisa Kendall (Thematix Partners) presented OMG’s work on knowledge representation, mostly done under the Ontology Platform SIGA lively panel, moderated by Claude, featured Elisa, Roger Burkhart (John Deere), and Artem Beer (Torch.AI).

AI PTF Meeting Introduction and Background

Claude Baudoin (cébé IT & Knowledge Management) and Bobbin Teegarden co-chair the AI PTF. Claude Baudoin led the meeting, held on Wednesday afternoon, which attracted over 20 participants.

Other Attendees
Mike Abramson	ASMG
JD Baker	Sparx Systems
Allison Barnard-Feeney	NIST
Artem Beer	Torch.AI
Cory Casanave	Model Driven Solutions
Fred Cummins	Agile Enterprise Design
Charlie Fudge	NSWC-DD
Koji Kamei	JARA
Elisa Kendall	Thematix
Simon Mettrick	BAE Systems
Takahiro Miyashita	JARA
Chokri Mraidha	CEA
Philip Newcomb	TSRI
Gerardo Pardo Castellote	RTI
Clayton Pummill	Torch.AI
Pete Rivett	Adaptive
Davide Sottara	Mayo Clinic
Hugues Vincent	Thales
Yong Xue	DOD/DISA

Claude gave a summary of the past events leading to this meeting:

In March, the OMG Board asked for the development of an AI standards strategy.On May 1, NIST issued an RFI on AI Standards. The strategy paper was set aside in order to devote time to a response, sent to NIST on June 6. NIST received 98 responses in total.The strategy paper development resumed in July and August, with help from Pete Rivett, Elisa Kendall, Sridhar Iyengar, and a few others. The paper benefited from the work done on the NIST RFI, and was completed and sent to the Board on August 26.Meanwhile, at the June meeting in Amsterdam, the Agent PSIG was “morphed” into an AI PSIG. However, it was the opinion of OMG management that a Task Force would be more visible, and they pushed for this to be proposed at the September meeting in Nashville.A “formation meeting” was held on September 25 and followed by a workshop on Augmented Reality. An AI PTF charter, published earlier, was approved by the PTC on September 27. A report from this meeting was sent to the ai@omg.org mailing list a week before this meeting, and is filed as OMG document ai/19-12-01.

Charter Update

Upon review of the charter, three changes were proposed and approved by the PTF, resulting in document ai/19-12-03. They were subsequently adopted during the PTC plenary session on Friday morning. Both changes related to bullets in the scope statement within the charter:

The third bullet, “image and sound recognition,” becomes “Image understanding, speech recognition, and computer vision”A new fourth bullet, “Robotic Systems,” was addedThe fifth (now sixth) bullet, “autonomous systems and agents,” becomes “autonomous and autonomic systems and agents.”

Brainstorm: Areas of Standardization in AI

The most common question we have heard -– and tried to answer -– so far is, “what is worth and useful to standardize in AI?” Behind that question lurks a suspicion that there are some standardization efforts that might have a stifling effect on innovation, and that we need to avoid this undesirable effect. The following ideas were collected (some topics were restructured for consistency, and footnotes were added for clarification):

A “notional AI architecture”See Ed Feigenbaum’s three books on AI -- presumably referring to Computers and Thought, the Handbook of Artificial Intelligence, and The Fifth Generation: Artificial Intelligence and Japan's Computer Challenge to the World.See what university courses describe.See the AAAI’s taxonomy of areas of AI -- this appears in an Appendix in the OMG AI Strategy paper. It is not an architecture, just a taxonomy, and it only appears in the form of a menu of search options within the AITopics publication, under the “Technology” facet (left column).It needs to document the various “paradigms of AI.”Integrity of systemsAuditability and bias (e.g., avoid bias in lending decisions, in social media filters, in medical advice engines…)Ability to check the consistency of decisionsTransparency -- this would include the “explainability” of machine learning algorithmsWhat kinds of AI are being used?Define clear requirements for bias avoidance, e.g. in terms of percentages of false positives or false negativesAPIsWhat do AI applications need from AI platforms? What APIs would make those AI applications portable across platforms?Find who is working on what in other OMG subgroups as well as outsideSome initiatives are listed in the AI strategy paperSee what else is suggested in the 97 other responses to the NIST RFI on AI StandardsModeling of AI ComponentsExtend the Architecture Modernization Task Force’s Abstract Syntax Tree Metamodel (ASTM) to AI languagesAlso extend CISQ’s metrics (adopted as OMG standards through the same ADM Task Force) to AI languages.Guidelines on where the biggest RoI for AI isA glossary of all the terms we have been usingThere is a Glossary of Artificial Intelligence in Wikipedia, don’t redo this work if it is satisfactorySee the disciplines and courses covered at MIT’s CSAIL (Computer Science and AI Lab)Something to allow the automatic security classification of dataSee Mike Abramson’s work on IEF/DCS (next section below)

AI and the Information Exchange Framework / Data-Centric Security (IEF/DCS)

Mike Abramson (ASMG) had asked to present a summary of the status of his work on IEF/DCS and the relevance of AI to it. His short slide set is document ai/19-12-02. IEF provides an architecture to manage the restrictions on interchange of confidential documents and e-mails, but the work of classifying an enterprise’s information assets in order to apply IEF policies to them is impossible to perform manually. Therefore, some automatic and intelligent classification process is going to be necessary.

Brainstorm: Future AI PTF Activities

This second brainstorm aimed at populating our roadmap for 2020 and beyond.

Liaison with Other Subgroups

As emphasized during the September meeting, the AI PTF needs to work closely with other OMG subgroups to (a) collect their requirements, (b) offer our help, (c) avoid duplicating their efforts or taking over what they may be already doing very well – especially, but not only, in the Ontology PSIG or the Robotics DTF. To this effect, we listed a number of target groups and their putative “correspondents” (some of them, identified with a question mark, were assigned in absentia) to ensure this coordination. The responsibilities we wish to assign to these correspondents are:

Help us get time on their own agendas to present what we are doingProvide the AI PTF with their groups’ priorities and requirements regarding AIConvey to their members any proposals we wish to make, so we can receive direct feedback.The list of proposed liaisons is as follows, with question marks for people who were not in the room and whose agreement has been assumed rather than verified.

OMG Subgroup	Liaison(s)
Architecture-Driven Modernization PTF	Philip Newcomb
Finance DTF	Mike Bennett
Business Modeling & Integration DTF	Fred Cummins
Ontology PSIG	Elisa Kendall (with Nick Stavros as backup)
Government Information Sharing DTF	Cory Casanave
Federated Enterprise Risk Management WG	Lars Toomre
ManTIS DTF	Uwe Kaufman (?)
Systems Engineering DSIG	Ed Seidewitz, Sandy Friedenthal or Conrad Bock (?)
C4I PTF	Simon Mettrick
Blockchain PSIG	Nick Stavros
Healthcare DTF	Davide Sottara
Retail DTF	John Glaubitz (?)
Robotics DTF	Koji Kamei
MARS PTF	Char Wales (?)
Analysis and Design PTF	Jim Logan (?)
Space DTF	Brad Kizzort (?)
Agent PSIG	Zack Hashemi (?)

Upcoming AI PTF Activities

This was a second brainstorm about activities we need to undertake –- not quite a roadmap yet, but aimed to lead us toward creating one.

Liaise with other OMG subgroupsSee the previous subsectionComplete the review of 98 responses to NIST’s RFI on AI Standards (97 excluding OMG's):Claude showed the existing Google Sheet, partially filled by Pete Rivett and him. We need more volunteers to (a) analyze the remaining responses, (b) review the existing analyses and provide additional opinions if desired.Add new content to our wikiList of upcoming conferencesMore referencesThe current page was created and maintained by Claude – we need more co-authors!Create a “reference model” or “landscape” of AIWe would love for NIST to do something similar to their authoritative “cloud computing reference model” of 2011. Allison Barnard-Feeney said that NIST is receiving a significant amount of money for its AI program, and is hiring right now to staff this.If we do something ourselves, we need to resolve the language -– is it a reference architecture, reference model, a landscape? Mike Bennett said that one size does not fit all. Claude described the shape of it as a “bento box.”Query universities and research centers on their current research projectsFind what’s new and exciting in AIStanford, MIT CSAIL, MIT Media Lab, Frankfurt (Roberto Zicari’s work on inspection of AI), SINTEF in Norway, and many more...Potentially issue a complementary RFI to NIST'sto ask additional questionsto involve non-US entities that might have been discouraged by NIST's questions aimed at ensuring or reinforcing the competitiveness of American companies.Transform our findings into discussion papers.At a later point in the meeting, we transformed this into a clearer roadmap for 2020:

When	What
Q1	Finish the analysis of the responses to the NIST RFI March meeting: visit with 3-4 other OMG subgroups Determine the need for an RFI, and define its content Plan and hold the Reston “AI Standards Forum”
Q2	June meeting: visit with 3-4 other OMG subgroups Collect results from OMG RFI (if we issued one) and from other consulted groups Create a first more substantial roadmap Select a tentative focus for a first RFP June meeting: joint event between the Finance DTF and AI PTF on AI for finance, with participation of the Boston Fed and State Street?
Q3	September meeting: visit with 3-4 other OMG subgroups Issue first RFP (note: Richard Soley would like this to happen by June) Adopt an AI “reference model” –- or other name for it
Q4	December meeting: visit with 3-4 other OMG subgroups Review the status of responses to the RFP (optimistic) Select a focus for a second RFP

Agenda for the March 2020 Meeting

There was a consensus that we should strive for a three-part agenda in Reston:

A regular half-day meeting of the Task Force, focused on our internal status review, planning, work on an RFP, etc.Joint sessions with other Task Forces and SIGS – for which we need to create a standard slide deck that any of us can take to those bodies and deliver a consistent messageA full-day “AI Forum” with papers and a panel:on a day selected in coordination with OMG’s MarCom team, since space is at a premium during our meetings in Reston,preceded by a Call for Papers that would specifically solicit presentations on AI landscapes, architectures, reference models, ontologies, etc.,heavily marketed to our contacts within NIST, the DoD, and other Washington area stakeholders.Philip Newcomb volunteered to form and lead a Program Committee for this event. In order to pull this off, we will need to have several regular conference calls between now and then. There are several topics that could be presented/discussed either at the smaller internal meeting or at the forum:

a presentation by Torch.AI,a discussion with the Joint AI Center (JAIC) facilitated through Steve MacLaird,a repeat presentation by Andreas Vogel on AI Ethics.

Summary of Action Items

In closing, we briefly recapitulated the action items that need to be executed in the near future. Lars Toomre suggested that we exchange the e-mail addresses of everyone who attended this meeting, so we know each other.

Bobbin asked how we should collect a list of conferences and activities to follow. Claude agreed to receive such messages and to add the suggestions to the wiki.

Given the need to coordinate the March schedule but also to act on OMG management's urgent desire to see a first RFP issued soon, we need to schedule teleconferences in January and February. Claude will “doodle” this.