Sidebar
This is an old revision of the document!
Table of Contents
4.2.10 Usability
About
Usability is defined by ISO/IEC 25010:2011 SQuaRE -- System and Software Quality Models as the degree to which a product or system can be used by Stakeholder (i.e., specified users) to achieve specified goals within a specified context.
Goals
The goals are1):
- Effectiveness - The accuracy and completeness with which users achieve specified goals
- Efficiency - The resources expended in relation to the accuracy and completeness with which users achieve goals.
- Satisfaction - The comfort and acceptability of use.
- Note: See also
Sub-Characteristics
This characteristic is composed of the following sub-characteristics2):
- Appropriateness Recognizability - Degree to which users can recognize whether a product or system is appropriate for their needs. (1)
- Learnability - Degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use. (1)
- Operability - Degree to which a product or system has attributes that make it easy to operate and control. (1)
- user_error_protection - Degree to which a system protects users against making errors. (1)
- user_interface_aesthetics - Degree to which a user interface enables pleasing and satisfying interaction for the user. (1)
- Accessibility - Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use. (1)
See:
Metrics
Usability as a characteristic is often considered a subjective quality and left to “interpretation”, however, there are metrics which use to quantify these sub-characteristics. Before we delve into the definition of the specific metrics, it is important to understand why we need metrics rather than just rely on intuitive evaluations.
A core reason to collect Usability Metrics is to provide a data about stakeholder's understanding about a product's usability rather than the developer's understanding of usability. When the two understandings (i.e., interpretations) converge everyone is happy and a way forward can result. That result may be to either continue in the same direction or to have a reassessment of the user's needs.
The metrics must quantify that the system meets the goals of the overall system:
- The Effectiveness Metrics of the communication between the system and the users
- The Efficiency Metrics of the users use of the system to accomplish their work
- The Satisfaction Metrics of the users that the sub-characteristics of the system are met.
Ultimately, the primary objective of usability metrics for evaluating a system or product is properly engineered (i.e., neither under- or over-engineered).
Satisfaction Metrics
Usability Metrics are generally done using through standardized questions designed to capture a the user's sentiments about the application, product or system. The survey's pose questions to the users and provide a scale of acceptability they user choses in assessing a particular attribute. The most common scale is based on the Likert Scales originally proposed in 1032 3).
Figure 1 gives a few of the Scales that Lickert defined. There are more available here:
| Scale | Attitude / Sentiment | ||||
|---|---|---|---|---|---|
| Agreement | Strongly Disagree | Disagree | Undecided | Agree | Strongly Agree |
| Frequency | Never | Rarely | Sometimes | Often | Always |
| Importance | Unimportant | Important | Moderately Important | Important | Very Important |
| Quality | Very Poor | Poor | Fair | Good | Excellent |
| Likelihood | Almost Never True | Usually Not True | Occasionally True | Usually True | Almost Always True |
| Score | 1 | 3 | 3 | 4 | 5 |
There are two ways that user satisfaction can be measured:
- Task Level Satisfaction - The Task Level Satisfaction is made at the end of each task attempted by the user. Note, a task may be attempted but it may not be completed. Therefore, it is important o record not just the attitude or sentiment about the task, but also the status of the task when the user takes the survey.
- Test Level Satisfaction - Similar to the Task Level Satisfaction, but Test Level Satisfaction is conducted at the end of a Test which can be comprised of multiple tasks. Therefore, in order to properly assess the Test Level, an evaluation of the Task assessments also needs to be made. For example, a test assessment might be low because some of the tasks were assessed as poor.
ISO also provides some guidance in how to assess User Satisfaction. See:
- Note: For more information, see: https://blog.ansi.org/2018/07/customer-satisfaction-iso-10002-quality/#gref
DDS Specifics Metrics
