Click Here to return to the main MBSE Wiki Page.

Methodology and Metrics

Purpose

The purpose of this MBSE activity in support of the OMG Systems Engineering DSIG and INCOSE MBSE Initiative is two fold. First, is to provide the international SE community with a current survey of some of the leading Model-Based Systems Engineering (MBSE) processes and methods used in industry today with specific emphasis on methodology and second, to develop useful metrics that can be used on MBSE-related programs and projects; more specifically, tool metrics and process metrics.

Measure of Success

SE practitioners utilize the information for tailoring one or more of the candidate MBSE methodologies into their own internal MBSE process architecture.

SE practitioners help define requirements for tool metrics and then share case studies how these metrics aided in their work and project reporting structure.

SE practitioners utilize process metrics defined as part of this activity during their MBSE-based programs and projects.

To help measure our success, go to the MBSE Benchmarking Survey below.

Topic Overview / Description

In general, a methodology can be defined as the collection of related processes, methods, and tools used to support a specific discipline [Martin, 1996]. That more general notion of methodology can be specialized to MBSE methodology, which we characterize as the collection of related processes, methods, and tools used to support the discipline of systems engineering in a “model-based” or “model-driven” context [Estefan, 2008].

In 2007, a formal survey of candidate MBSE methodologies was published as part of the work of the INCOSE MBSE Focus Group that later was formalized as the INCOSE MBSE Initiative. In 2008, that formal survey of candidate MBSE methodologies was published under the auspices of an INCOSE technical publication [see Estefan, 2008 under INCOSE Links below]. The 2008 report surveyed six (6) candidate MBSE methodologies: INCOSE Object-Oriented Systems Engineering Method (OOSEM), IBM Rational Telelogic Harmony-SE, IBM Rational Unified Process for Systems Engineering (RUP-SE), Vitech MBSE Methodology, JPL State Analysis (SA), and Dori Object-Process Methodology (OPM). It should be noted that the scope of the report went beyond a simple survey and also documented a number of key issues related to the discipline of MBSE, including the following: differentiating processes, methods, and tools; characterizing the role of lifecycle models (project, acquisition, and systems engineering); an explanation of models in support of MBSE processes; and, documenting the roles of model-based hazard analysis, UML/OMG SysML, OMG model-driven architecture (MDA), and executable UML foundation.

Although the 2008 MBSE survey report is a publically-accessible INCOSE technical publication (i.e., does not require INCOSE membership to access), in an effort to better serve the general population of MBSE practioners who may not have access to the myriad of other the work products of the INCOSE MBSE Initiative; and, in an effort to keep information current, a brief summary of the candidate methodologies surveyed in that 2008 report as well as additional methodologies that have been identified as gaps since that publication are described herein. The benefit over updating the INCOSE technical publication is that the lead methodologists can update this public Wiki page on an as needed basis to reflect the most current data for a particular methodology.

Because methodologies embody a set of process-based best practices and assets, methodology and metrics are closely related activities. There are at least three reasons for collecting process metrics [Martin, 1996]. First, a process must be measured in order to improve it. Second, metrics provide project data for cost estimating and for planning the required activities and schedule intervals. Third, metrics provide a benchmark against which it is possible to compare performance against other projects and organizations. A methodology will more likely be adopted for use if there are metrics available to prove actual benefits [Estefan, 2008]. An excellent resource on metrics is the INCOSE Metric Guidebook for Integrated Systems and Product Development [Wilbur et al., 2005].

Schedule

DateMilestoneStatusPoint of Contact
IW11Summary of Current MBSE Methodologies Listed & References ProvidedCompleteJeff

Team Members

NameOrganizationContact Information
Jeff A. Estefan NASA/JPL Jeffrey.A.Estefan@jpl.nasa.gov
Michelle Sprecht IBM michelle.specht@us.ibm.com
John C Watson (Lead) Lockheed Martin (Retired) jcwatson@ieee.org
J.D. Baker No Magic james.baker@incose.org

MBSE Methodology

Definitions

  • Process - A logical sequence of tasks performed to achieve a particular objective. A process defines the “WHAT” is to be done, without specifying the “HOW” each task is to be performed.
  • Method - Consists of techniques for performing a task, the “HOW” of each task. The terms “method,” “technique,” practice,” and “procedure” can be used interchangeably in this context.
  • Tool - An instrument that, when applied to a particular method, can enhance the efficiency of a task. Thus, methods help bridge the gap between process and tools. The purpose of the tool should be to facilitate the accomplishment of the “HOWs”.
  • Methodology - Defined as a collection of related processes, methods, and tools.

List of Methodologies and Methods

Methodologies Surveyed in INCOSE 2008 Report

Additional Methodologies Identified as Gaps Since 2008 INCOSE Survey

MBSE Metrics

MBSE ROI Metric Papers Realized from Industry

Link to MBSE Metric Papers Realized from Industry

In order to justify the change and investment to a model-based System Engineering approach the industry is asking for quantitative evidence that reveals the return on investment (ROI). For various reasons this quantitative evidence, or metrics, has been difficult to come by. Often times this information is sensitive or proprietary and therefore cannot be shared, or in many transitions to model-based the metrics simply were not collected. In addition, sometimes the metrics collected are questioned or minimized because it is difficult to make an “apples to apples” comparison.

The intent of this set of wiki pages, available via of the link below, is to be a collection point for a set of papers, web references, books and summary information documenting both quantitative and/or anecdotal model-based metrics experiences realized from across the industry. The intent is not just to bring to the surface the “good news” but also to include lessons learned from both positive and less than positive experiences.

If you know of any additional information available to share to the industry, please forward the reference to john.watson@lmco.com and we will include it in the survey.

Metric Tools

This link shows an example of a Metric Tool Dashboard.

INCOSE Events

MBSE International Workshop at INCOSE IW 2013, Jacksonville Fl, Jan 26th and 27th

MBSE International Workshop at INCOSE IW 2012, Jacksonville Fl, Jan 21st and 22nd

MBSE International Workshop at INCOSE IW 2011, Phoenix Az

Reference Links

Link to related items such as related organizations, related articles, …

MBSE Benchmarking Survey

To help us collect project metrics relative to your experiences using MBSE please download and complete the following questionair Voluntary MBSE Questionaire and forward your responses to Jason Kruska at Jason.Kruska-1@nasa.gov. Thank you!

Methodology and Metrics Reference Materials

INCOSE Links

Estefan, Jeff A., “Survey of Model-Based Systems Engineering (MBSE) Methodologies,” Rev. B, INCOSE Technical Publication, Document No.: INCOSE-TD-2007-003-01, International Council on Systems Engineering, San Diego, CA, June 10, 2008.

Wilbur, Ann, Towers, Gayle, Sherman, Tom, Yasukawa, Dan and Sue Shreve, ”Metrics Guidebook for Integrated Systems and Product Development,” INCOSE Technical Publication, Document No.: INCOSE-TP-1995-002-01, International Council on Systems Engineering, San Diego, CA, July 25, 2005.

OMG Links

External Links

Martin, James N., Systems Engineering Guidebook: A Process for Developing Systems and Products, CRC Press, Inc.: Boca Raton, FL, 1996.

Cross Team Links

RETURN to MBSE Wiki Home Page

Click Here to return to the main MBSE Wiki Page.

 
mbse/methodology.txt · Last modified: 2015/12/03 16:27 by jcwatson
 
OMG Home Logos and Trademarks Become a Member Become a Sponsor Upcoming TC Meeting TOP