Why Everyone Should Run Remote Usability Studies

If you are reading this book, then you probably already know what usability testing is, but we don't want to make any assumptions about that, which takes us right to the essence of this book: usability testing is about not making assumptions. In fact, it takes the opposite approach. Usability studies are executed in order to gain concrete, actionable insights. Instead of assuming that users will use an interface in a certain manner, usability tests are run to actually monitor their use of the interface, identify where they stumble and what they appreciate, hear their thoughts, understand their decisions and ultimately use this information to improve the product.

Let's take a step back and talk about usability. Usability refers to how easy an interface is to use. It is a quality that every interface inherently possesses. Issues arise when this quality is not very pronounced. ISO 9241-11 defines usability as the "extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use".

Effectiveness (how well the user achieves their goal) and efficiency (how much effort is required from the user to achieve their goal) are unsurprising in the ISO description. Satisfaction, however, is interesting: even if an interface is effective (meaning that it does what it needs to do) and efficient (meaning that it does not require a lot of effort to do it), how the user feels about using the interface is also relevant. For every app someone uses, there is probably a plethora of competing apps that do exactly the same thing. So how does the user choose? Oftentimes, they go by the very subjective feeling of being satisfied or not. The evaluation of the usability of an interface is therefore based on both very objective measures, such as the task completion rate or the time spent on a task, but also on the very subjective perception of the individual user using the interface. Did the user like the design? Are the colors pleasing? Does the interface make the user feel good about themselves? Understanding what users like or dislike, where they struggle, and which tasks come easy to them, helps with the following:

• Evaluate the usability of the interface
• Identify areas to fix and patterns to avoid
• Determine a usability baseline across product iterations
• Compare an interface with the competition

A more formal approach to evaluate the usability of a product is to run a usability test. Usability testing refers to "evaluating a product or service by testing it with representative users" (https://www.usability.gov/how-to-and-tools/methods/usability-testing.html). Usability testing is a subdiscipline of User Experience (UX). Its goal is to ensure that a given product is easy to use and that the user's experience with the product is intuitive, useful, and satisfying. Essentially, users, who are representative of the target users, are monitored while using an interface to complete tasks that are relevant to their needs. A usability test can be run at any stage of the product development lifecycle. Usability tests in the early design stages can be used to validate a navigation concept using wireframes, for example, or test the usability of new features using early prototypes, or benchmark the ease-of-use of the final product. The feedback is then used to refine and improve the user interface, ideally in an iterative process.

We want to point out that usability testing is not market research. While usability testing is mostly focused on the interaction with a particular product, market research is usually less specific; usability testing is about ease-of-use, whereas market research is mostly about user opinions or past experience. Usability testing does not require a large number of participants in order to generate valuable results, whereas most market research tools depend on large, statistically relevant sample sets.

Usability testing is also not Quality Assurance (QA). Quality assurance is performed by qualified testers, whereas usability testing is preferably run with non-QA testers. Quality assurance is aimed at finding bugs, whereas usability testing is preferably run on bug-free implementations. Usability testing usually involves a user completing a task the way they normally would, while quality assurance testers will repeat that same task multiple times in order to mimic every possible permutation a real user might encounter.