User Tools

Site Tools


mbse:metricreferences

MBE Metric Papers Realized from Industry

The intent of this wiki page is to provide a set of links to papers, web references, books and summary information documenting quantitative and/or anecdotal metrics realized from industry development efforts.

Title Reference Type Citation Additional Information
Final Report- Model Based Engineering (MBE) Subcommittee Presentation Bergenthal, Jeff. Final Report- Model Based Engineering (MBE) Subcommittee. NDIA Systems Engineering Division M&S Committee. February 2011.The presentation provides the final report of the NDIA Model Based Engineering Subcommittee. It provides potential MBE benefits, costs and risks, the current and to-be MBE state and some recommendations for going forward. In addition it provides a large collection of MBE related papers, see slides 33 through 37. On chart 11 it references some of those papers to the acquisition life cycle.
AVSI’s SAVI Program: Proof of Concept Demonstrations Presentation Ward, Don. AVSI’s System Architecture Virtual Integration Program: Proof of Concept Demonstrations. INCOSE IW 2013 MBSE Workshop. January 2013.This presentation provides a summary description of the AVSI’s (Aerospace Vehicle Systems Institute) Systems Architecture Virtual Integration (SAVI) Proof of Concept project, it provides actual measured return on investment results of the project and it provides the next set of planned steps for the project.
How Product Development Organizations can Achieve Long-Term Cost Savings Using Model-Based Systems Engineering (MBSE) Paper Reference Krasner, Jerry. How Product Development Organizations can Achieve Long-Term Cost Savings Using Model-Based Systems Engineering (MBSE). October 2015.A report by Embedded Market Forecasters (EMF), the market research division of American Technology International, Inc., reports measured improvements as a result of MBSE. Their data set is based on 6 years of survey data totaling more than 4000 developer responses, and claims 95% +/- 3% statistical accuracy of conclusions from their 2015 data set. In summary, (1) MBSE delivers a 55% reduction in total development cost; (2) MBSE enables product-line engineering, which delivers a 42% reduction in cost compared to typical SE; (3) MBSE reduces design delays and cancellations, improves the relationship between pre-design expectations and final designs, enables faster completion of products, enables easy upgrades to meet new market opportunities, results in minimal redesign necessitated by changes in available hardware, minimizes the possibility of product recalls, and makes in-field support more cost effective because support personnel can more clearly understand design models and code.
Cost of Software Defects Web Reference Strickler,Jon. Cost of Software Defects. Agile Elements. April 22, 2008. This web site provides metrics associated with defects found in different phases of the development cycle.
Toward Model-Based Embedded System Validation through Virtual Integration Web Reference Feiler, P. H. & Hansson, J. Toward Model-Based Embedded System Validation through Virtual Integration. SoftwareTech. Vol. 12, No.4. 2010. This paper describes the use of a model-based approach to reduce defects and identify defects earlier.
Toward Model-Based Embedded System Validation through Virtual Integration Web Reference Feiler, P. H. & Hansson, J. Toward Model-Based Embedded System Validation through Virtual Integration. SoftwareTech. Vol. 12, No.4. 2010. This paper describes the use of a model-based approach to reduce defects and identify defects earlier.
IBM Rhapsody Case Studies Presentation Software and Systems Engineering, IBM Rational. 2011. This presentation from IBM provides quantitative ROI information from over 20 companies that use their tools for model based development. Sheets 4 and 5 presents a summary of the Qualified ROI metrics collected from these companies.
Modeling Approach for Re-designing the Mission Operations System for Human Missions Paper Reference Meshkat, Pennington, Ruszkowski, Scott, Manno.A Modeling Approach for Re-designing the Mission Operations System for Human Missions. NASA JPL. 2010. This paper describes NASA’s systems engineering approach for re-engineering the Mission Operations System (MOS) using advanced modeling and analysis techniques. This paper explains the efforts undertaken by the NASA team, the challenges faced and the lessons learned to date.
Ratheon at IMB Innovate 2012 Video Wells, Brian. youTube Video, Raytheon at IBM Innovate 2012. June 2012. This video provides a Raytheon perspective on the benefits of model based engineering
NDIA Paper:Use of a Model-Based Approach to Minimize System Development Risk and Time-to-Field for New Systems Presentation Wagner, Brockwell, Daniels, Loesh, Gosnell. NDIA Paper:Use of a Model-Based Approach to Minimize System Development Risk and Time-to-Field for New Systems. NDIA. Oct. 9, 2010. This presentation identifies six root problems and describes how an integrated MBSD environment can help. It provides a set of metrics captured from an actual project.
Survey of Model-Based Systems Engineering (MBSE) Methodologies Paper Reference Estefan, Jeff A. Survey of Model-Based Systems Engineering (MBSE) Methodologies. Survey of Model-Based Systems Engineering (MBSE) Methodologies. May 25, 2007. This paper identifies some industry leading methodologies used for MBSE and provides a deciption of each. Some of thes methodology describtions provide anecdotal metrics as to their value added.
Model-Based Development of Neural Prostheses for Movement Paper Reference Davoodi, Rahman., et al. Model-Based Development of Neural Prostheses for Movement. IEEE Transactions on Biomedical Engineering, Vol. 54, No. 11. Nov. 2007.This paper describes an integrated model-based approach to use virtual prototyping to simulate mechanical behavior before manufacturing and discusses the benefits of such an approach.
On the Unification Power of Models Paper Reference Bézivin,Jean. On the Unification Power of Models. ATLAS Group. 2003.This paper proposes a vision of the development of MDE based on some lessons learnt in the past 30 years in the development of object technology.
A Practical Guide to SysML: The Systems Modeling Language Book Reference Friedenthal, Sandford, et al. A Practical Guide to SysML: The Systems Modeling Language. Morgan Kaufmann. Chapter 2. Sept. 3, 2009. This book provides a number of references to the types of metrics that can be collected in a model-based environment, and how they can be used to improve the quality and measure progress. Refer to the book’s index under “metrics”.
Motivation for Model Driven Development Paper Reference Watson, John. Motivation for Model Driven Development . June 2008. This paper provides a list of motivations for using a model driven approach for developing large, complex software intensive systems. This document will attempt to communicate why we need to embrace these techniques across the industry.
"Are We There Yet?" Assessing Quality in Model Based Systems Engineering Paper Reference Matthew Hause. “Are we there yet?” Assessing Quality in Model Based Systems Engineering. INCOSE. 2011. This paper examines a variety of different criteria for quality in model-based development and demonstrates some of the ways quality can be defined and evaluated in a model.
Existing Model Metrics and Relations to Model Quality Paper Reference Mohagheghi, Parastoo, Dehlen, Vegard . Existing Model Metrics and Relations to Model Quality. OMG. This paper presents quality goals for models and provides a state-of-the-art analysis regarding model metrics.
Evaluating Quality in Model-Driven Engineering Paper Reference Mohagheghi, Parastoo, Aagedal, Jan. Evaluating Quality in Model-Driven Engineering. OMG.This paper presents quality goals in MDE and states that the quality of models is affected by the quality of modeling languages, tools, modeling processes, the knowledge and experience of modelers, and the quality assurance techniques applied.
EDS - Concurrent Design Facility Presentation The ESA Concurrent Design Facility: Concurrent Engineering applied to space mission assessments . CDF ESA/ESTEC Noordwijk – NL . 2011. The ESTEC Concurrent Design Facility is an Integrated Design Environment (IDE) available to all ESA programs for interdisciplinary and inter-directorate applications, based on Concurrent Engineering methodology. The intent is to overcome the communication gaps between the “designer” (who produces design information) and the “user” (who utilizes the design information). This presentation provides quantitative and anecdotal metrics for using this MBE review approach.
Making model-based code generation work Paper Reference Tolvanen, Juha-Pekka. Making model-based code generation work. EmbeddedSystems Europe. Aug./Sept. 2004.
Model-Based System Engineering (MBSE) Organizational Panacea or Mirage Presentation Wasson, Charles. Model-Based System Engineering Organizational Panacea or Mirage. INCOSE Great Lakes Conference. Nov. 2011.
Correlating Product And Process Measures Paper Reference Carson, Ronald S., Correlating Product and Process Measures as a Model for Systems Engineering Measurement, Proceedings of the International Council on Systems Engineering 2011. The failure to establish product measures and provably correlate them with process measures leads to open-loop management systems that can only measure process compliance, with unknown effect on product quality. In this paper a set of standard categories of measures are applied to various Systems Engineering work products identified as “output” in the INCOSE SE Handbook. Process and product measures for each process output are proposed, along with correlations between them, and associations to the INCOSE SE Leading Indicators where applicable.
A Practical Program of Research to Measure Systems Engineering Return on Investment (SE-ROI) Paper Reference Honour, Eric C., A Practical Program of Research to Measure Systems Engineering Return on Investment (SE-ROI), Published and used by INCOSE with permission. This paper provides insight as to the correct amount of SE effort to apply to a program. An ongoing project, Systems Engineering Return on Investment (SE-ROI), aims to quantify the correlations by gathering data on current and completed programs. This paper describes the practical program of research being used in the SE-ROI project and the current state of that development.
Return of experience on the implementation of the System Engineering approach in Alstom Presentation Ferrogalini, Marco and Le Bastard, Jean. Presented at the Complex Systems Design & Management 2012, Paris France.This presentation introduces a more robust system approach for railway systems. It defines system views for a top-down model based approach starting with requirements and using a SysML model to create the various views. The presentation also states the gained benefits and system engineering team feedback (slides 9-11).
A World in Motion – Systems Engineering Vision 2025 Presentation Beihoff, Friedenthal, Kemp, Nichols, Oster, Paredis, Stoewer, Wade. A World in Motion – Systems Engineering Vision 2025. Published by the International Council on Systems Engineering 2014. Vision 2025 is a publication produced by a team of leaders from industry, academia and government with the intent that it will be used by people working in many domains – healthcare, utilities, transportation, defense, finance – who will add their unique perspectives to the role systems engineering plays in our serving our world’s many complicated demands.
NASA/JPL Symposium & Workshop on Model-Based Systems Engineering Presentation Collection Jet Propulsion Laboratory, California Institute of Technology, 2015. NASA/JPL hosted a symposium and workshop on model-based systems engineering (MBSE) on January 28-30, 2015. The theme was “Moving MBSE into Practice”. This link provides a series of excellent presentations about the state of the practice, challenges, recommendations, future directions and strategies.
Developing a Strategy and Roadmap for Advancing the State-of-the-Practice of MBSE Presentation Estefan, Jeff A., Developing a Strategy and Roadmap for Advancing the State-of-the-Practice of MBSE within Your Organization. Jet Propulsion Laboratory, California Institute of Technology 2010. Transitioning from the traditional document-based approach to systems engineering to a model-based approach remains difficult for most organizations. This presentation provides insight for how to plan and execute the transition to a model based approach.
Model based Systems Engineering (MBSE) Missing Link in the digital Enterprise Strategy Presentation Stoewer, Heinz, Model based Systems Engineering (MBSE) Missing Link in the digital Enterprise Strategy ?. Presented at the MBSE Workshop at the INCOSE Los Angeles International Workshop, January 25, 2014. Keynote to the MBSE Workshop at the INCOSE Los Angeles International Workshop. The presentation discusses some history of Systems Engineering; the emerging digital world; the Systems Engineering World in context; the MBSE status and future; and provides a summary and a few take a ways.
Help for Systems of Systems Using UPDM and RAS for SoS Modeling Paper Reference Hause, Matthew C., Help for Systems of Systems Using UPDM and RAS for SoS Modeling. Published and used by INCOSE with permission, 2013. Systems of Systems (SoS) modeling is becoming important in both civilian and military systems. This paper shows how combining UPDM and RAS provides a Model of Models approach with the main model specifying assets in various levels of detail. The paper also includes an assessment of the applicability and effectiveness of this approach. The INCOSE System of Systems Working Group has collected a set of “Pain Points on SoS”. This paper provides review these pain points and discusses how the application of a Model of Model approach can help to alleviate them.
Building for Tomorrow: Towards 21st Century Systems Engineering Presentation Long, David., Building for Tomorrow: Towards 21st Century Systems Engineering. Published and used by INCOSE with permission, 2015 This presentation provides insight as to the future of Systems Engineering, what some of the challenges we face today and how to overcome some of those challenges. The following link provides a document with a YouTube link and chart timeline of the YouTube recording.

Return to MBSE Metrics

mbse/metricreferences.txt · Last modified: 2016/05/05 17:23 by jcwatson