This is an old revision of the document!
Usability is defined by ISO/IEC 25010:2011 SQuaRE -- System and Software Quality Models as the degree to which a product or system can be used by Stakeholder (i.e., specified users) to achieve specified goals within a specified context.
The goals are1):
This characteristic is composed of the following sub-characteristics2):
See:
Usability as a characteristic is often considered a subjective quality and left to “interpretation”, however, there are metrics which use to quantify these sub-characteristics. Before we delve into the definition of the specific metrics, it is important to understand why we need metrics rather than just rely on intuitive evaluations.
A core reason to collect Usability Metrics is to provide a data about stakeholder's understanding about a product's usability rather than the developer's understanding of usability. When the two understandings (i.e., interpretations) converge everyone is happy and a way forward can result. That result may be to either continue in the same direction or to have a reassessment of the user's needs.
The metrics must quantify that the system meets the goals of the overall system:
Ultimately, the primary objective of usability metrics for evaluating a system or product is properly engineered (i.e., neither under- or over-engineered).
Efficiency is measured in terms of the time it takes to complete a task from when the task is initiated to when it is successfully completed. The units used to record the time must be uniform for all tasks (i.e., milliseconds, seconds, minutes, etc).
Mathematically The time taken to complete a task can then be calculated by simply subtracting the start time from the end time:
There are two ways to calculate Efficiency:
In this calculation, the quotient from the division of the success of a task (either one or zero) divided by the time to accomplish a task is an indicator of the efficiency of the task. For example, if a tasks was not successful, then the success of the task is zero and the efficiency is zero. If the task is successful and it takes one minute to accomplish the task, then the efficient is one (i.e., 1/1). If it takes two minutes to accomplish the task, then the efficiency is one half (i.e., 1/2 = .5).
To calculate the Time-Based Efficiency for all tasks and all users, the following equation applies:
Where:
Justin Mifsud1 provides an excellent example of how for calculating time-based efficiency:
Suppose there are 4 users who use the same product to attempt to perform the same task (1 task). 3 users manage to successfully complete it – taking 1, 2 and 3 seconds respectively. The fourth user takes 6 seconds and then gives up without completing the task.
Taking the above equation:
Placing the above values in the equation:
The overall relative efficiency uses the ratio of the time taken by the users who successfully completed the task in relation to the total time taken by all users. The equation can thus be represented as follows3):
Justin Mifsud1 provides an excellent example of how for calculating Overall Relative Efficiency.
Assume there are 4 users who use the same product to attempt to perform the same task (1 task). 3 users manage to successfully complete it – taking 1, 2 and 3 seconds respectively. The fourth user takes 6 seconds and then gives up without completing the task.
Taking the above equation:
Placing the above values into the equation yields the following:
Usability Metrics are generally done using through standardized questions designed to capture a the user's sentiments about the application, product or system. The survey's pose questions to the users and provide a scale of acceptability they user choses in assessing a particular attribute. The most common scale is based on the Likert Scales originally proposed in 1032 4).
Figure 1 gives a few of the Scales that Lickert defined. There are more available here:
Scale | Attitude / Sentiment | ||||
---|---|---|---|---|---|
Agreement | Strongly Disagree | Disagree | Undecided | Agree | Strongly Agree |
Frequency | Never | Rarely | Sometimes | Often | Always |
Importance | Unimportant | Important | Moderately Important | Important | Very Important |
Quality | Very Poor | Poor | Fair | Good | Excellent |
Likelihood | Almost Never True | Usually Not True | Occasionally True | Usually True | Almost Always True |
Score | 1 | 3 | 3 | 4 | 5 |
There are two ways that user satisfaction can be measured:
ISO also provides some guidance in how to assess User Satisfaction. See: