4.2.10 Usability

Return to Non-Functional Requirements

About

Return to Top

Usability is defined by ISO/IEC 25010:2011 SQuaRE -- System and Software Quality Models as the degree to which a product or system can be used by Stakeholder (i.e., specified users) to achieve specified goals within a specified context.

Goals

The goals are1) Justin Mifsud, Usability Metrics – A Guide To Quantify The Usability Of Any System, Accessed 18 November 2020, https://usabilitygeek.com/usability-metrics-a-guide-to-quantify-system-usability/ :

Effectiveness - The accuracy and completeness with which users achieve specified goalsEfficiency - The resources expended in relation to the accuracy and completeness with which users achieve goals.Satisfaction - The comfort and acceptability of use.Note: See also

Sub-Characteristics

This characteristic is composed of the following sub-characteristics2) International Organization for Standardization (ISO), Usability, ISO25000, Accessed: 17 November 2020, https://iso25000.com/index.php/en/iso-25000-standards/iso-25010/61-usability :

Appropriateness Recognizability - Degree to which users can recognize whether a product or system is appropriate for their needs. (1)Learnability - Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use. (1)Operability - Degree to which a product or system has attributes that make it easy to operate and control. (1)user_error_protection - Degree to which a system protects users against making errors. (1)user_interface_aesthetics - Degree to which a user interface enables pleasing and satisfying interaction for the user. (1)Accessibility - Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use. (1)See:

Metrics

Return to Top

Usability as a characteristic is often considered a subjective quality and left to “interpretation”, however, there are metrics which use to quantify these sub-characteristics. Before we delve into the definition of the specific metrics, it is important to understand why we need metrics rather than just rely on intuitive evaluations.

A core reason to collect Usability Metrics is to provide a data about stakeholder's understanding about a product's usability rather than the developer's understanding of usability. When the two understandings (i.e., interpretations) converge everyone is happy and a way forward can result. That result may be to either continue in the same direction or to have a reassessment of the user's needs.

The metrics must quantify that the system meets the goals of the overall system:

The Effectiveness Metrics of the communication between the system and the usersThe Efficiency Metrics of the users use of the system to accomplish their workThe Satisfaction Metrics of the users that the sub-characteristics of the system are met.Ultimately, the primary objective of usability metrics for evaluating a system or product is properly engineered (i.e., neither under- or over-engineered).

Satisfaction Metrics

Return to Top

Usability Metrics are generally done using through standardized questions designed to capture a the user's sentiments about the application, product or system. The survey's pose questions to the users and provide a scale of acceptability they user choses in assessing a particular attribute. The most common scale is based on the Likert Scales originally proposed in 1032 3) Saul McLeod, Likert Scale Definition, Examples and Analysis , Simply Psychology, 2019, Accessed 20 November 2020, https://www.simplypsychology.org/likert-scale.html .

Figure gives a few of the Scales that Lickert defined. There are more available here:

Scale	Attitude / Sentiment
Agreement	Strongly Disagree	Disagree	Undecided	Agree	Strongly Agree
Frequency	Never	Rarely	Sometimes	Often	Always
Importance	Unimportant	Important	Moderately Important	Important	Very Important
Quality	Very Poor	Poor	Fair	Good	Excellent
Likelihood	Almost Never True	Usually Not True	Occasionally True	Usually True	Almost Always True
Score	1	3	3	4	5

The Lickert Scale

There are two ways that user satisfaction can be measured:

Task Level Satisfaction - The Task Level Satisfaction is made at the end of each task attempted by the user. Note, a task may be attempted but it may not be completed. Therefore, it is important o record not just the attitude or sentiment about the task, but also the status of the task when the user takes the survey.Test Level Satisfaction - Similar to the Task Level Satisfaction, but Test Level Satisfaction is conducted at the end of a Test which can be comprised of multiple tasks. Therefore, in order to properly assess the Test Level, an evaluation of the Task assessments also needs to be made. For example, a test assessment might be low because some of the tasks were assessed as poor. ISO also provides some guidance in how to assess User Satisfaction. See:

Note: For more information, see: https://blog.ansi.org/2018/07/customer-satisfaction-iso-10002-quality/#gref

DDS Specifics Metrics

Return to Top