User Tools

Site Tools


Sidebar

Welcome to DIDO WIKI

dido:public:ra:1.4_req:3_assessment:2_nonfunctional

4.4.2 Non-functional Requirements Assessment

Assessing the Alternatives

Return to Top

Non-Functional requirements are by nature, hard to measure and hard to assess for compliance. The complexity of the assessment problem is compounded because products or systems as well as the environment they operate within are not static. Therefore, an assessment that is done today, may no longer be accurate in the future. When assessing Commercial Off-The-Shelf (COTS), Government Off-The-Shelf (GOTS), Modified Off-The-Shelf (MOTS) or NATO Off-The-Shelf (NOTS) or the inclusion of Open Source Software (OSS) the potential for changes (particularly enhancements) needs to be assed also.

One way to accomplish this is to provided an assessment which is weighted for the “ease of implementation” for new features. For example, a system or product may not have done much to support 4.3.4.1 Confidentiality, but the vendor of the product must determine that it is an easy upgrade to add it to the product. On the other hand, the support for confidentiality might be extremely difficult. Sometimes, the feature may be easy to solve but requires time and money to accomplish. As a potential stakeholder, they can direct resources to help overcome this shortfall.

Figure 1: An overview of the Assessment Process

The Vendor's Assessment

Return to Top

The Vendor's assessment team must discuss with each other and use the DIDO-RA workbook to determine the weight of the requirement.

Figure 2: The Vendor's Assessment Team Activities

This is done by weighing each property, related to the Requirement, by giving a score between 1-100. 1 representing an easy development process, and 100 being impossible now, merely very difficult in a couple years.

Publish the assessment, and notify the vendor..

The Stakeholder's Focus Group

Return to Top

The Vendor's stakeholder assessment team must create a focus group of stakeholder's. This group is used in conjunction with the stakeholder assessment team to determine the importance of the Requirement and potentially direct more or less resources to this area.

Figure 5: The Stakeholder's Focus Group Activities

This is done by rating each property related to the requirement with either a '+' option representing a 'more important' status, or a '-' option representing a 'less important' status. On the Coefficient sheet choosing a '+' will cause a '-' to be put in the opposite cell in the table. EX: cell(1,2)='+' means cell(2,1)='-'. Also properties can't be compared to themselves, these cells have lines drawn through them to represent this.

At the end of filling out each table. Each row is counted for '+', the number of '+' in a row is equal to the coefficient for that property.

When completed publish the assessment and notify the stakeholder's.

The Stakeholder's Assessment Team

Return to Top

Once the assessments and importance coefficients are set, the DIDO-RA is passed to the stakeholder's assessments team. Here they must take the factor weighting and the coefficients to create the Figure of Merit (FoM). These figures are then published to assessment reports that are given to the Vendor, and/or given back to the stakeholder's assessment team for further review.

Figure 8: The Stakeholder's Assessment Team Activities

In the DIDO-RA workbook after the Coefficient/Factor Weight sheets are filled out, those values associated with the requirement, are then sent to the corresponding Requirement Sheets in the workbook.

This data is then used to create the FOM(Figures of Merit). The formula to calculate the FOM = (C1)(FW1) + (C2)(FW2)+ (Cn)(FWn) for each property. These Property FOMs are then added into a Requirement FOM.

NOTE: Comparing Requirement FOMs from one vendor to another will cause a skewed comparison. To compare between vendors you must break the Requirement FOMs into its components. The components of Requirement FOM are the Property FOMs.

The Stakeholder's Award Team

Return to Top

The stakeholder's receive the Assessment Reports as well, if they deem the results satisfactory they will choose the product just reviewed.

Figure 11: The award is usually your software is chosen to be used.
dido/public/ra/1.4_req/3_assessment/2_nonfunctional.txt · Last modified: 2021/08/17 14:40 by murphy
Translations of this page: