“Five Users are Enough to Catch 85% of the Usability Problems” — and Other Controversial Beliefs About Usability Testing

Short summary: This 45-minute talk presents a number of beliefs about usability testing and discusses whether they are myths or truths.

Summary: Over the years a number of simple, catchy and convenient beliefs about how to plan and conduct usability testing have emerged from hearsay, courses, articles and even textbooks. Unfortunately, closer research has often shown that some of these beliefs are oversimplifications or even incorrect, and that they may mislead professionals seriously, hiding an often complex or inconvenient truth.

This talk takes a critical look at some of these beliefs about usability testing. The talk discusses these beliefs and demonstrates that some of them are unsubstantiated myths or pure fantasy, and what the truth is.

Some of the controversial beliefs that will be discussed in this talk are:

  • The main goal of a usability test is to discover usability problems
  • Expert reviews provide results that are as reliable as those from usability tests
  • At least 25% of the comments in a usability test report should be positive

Audience participation is important for this talk. Attendees will be able to participate actively by voting “myth” or “fact” on statements using their smartphone. Attendees will also be able to participate in the discussion of each belief.

A video recording of this talk is available (35:52)

Target group: The talk is aimed at usability professionals who have some experience with usability testing, for example from moderating or observing usability tests.