Dialog Design DialogDesign ved Rolf Molich English English Dansk Dansk
DialogDesign, Skovkrogen 3, 3660 Stenløse, Denmark, dialogdesign@dialogdesign.dk, +45 4717 1731
Free Resources CUE Products Events Books About DialogDesign
CUE Papers CUE-1 CUE-2 CUE-3 CUE-4 CUE-5 CUE-6 CUE-7 CUE-8 CUE-9

CUE-4 - Hotel Pennsylvania

CUE-4 is a comparative usability evaluation of Hotel Pennsylvania’s website, www.hotelpenn.com, conducted in March 2003.

Seventeen professional teams simultaneously and independently evaluated the website’s usability. Nine teams used usability testing, and eight teams used their favorite inspection technique.

Papers about CUE-4

Joseph S. Dumas, Rolf Molich, and Robin Jeffries, “Describing Usability Problems - Are We Sending the Right Message,” 
Interactions, July/August 2004, pp. 24-29.
Abstract and e-prints

Rolf Molich and Joseph S. Dumas,
“Comparative Usability Evaluation (CUE-4),”
Behaviour & Information Technology
, Vol. 27, issue 3, 2008.
Abstract and e-prints

Rolf Molich, Robin Jeffries, and Joseph S. Dumas,
“Making Usability Recommendations Useful and Usable,”
Journal of Usability Studies
, vol. 2, no. 4, August 2007.
Abstract and e-prints

Available Downloads

·       The proposal for the CHI2003 workshop, which describes the study’s background and rules (PDF, 23 KB).

·       All 17 test reports in one PDF file (PDF, 3,823 KB).

·       List of participating teams (PDF, 6 KB).

·       Client test scenario that each team received, describing the usability test task (PDF, 47 KB).


The Most Important CUE-4 Findings

·      Usability testing isn’t the “high-quality gold standard” against which all other methods should be measured. CUE-4 shows that usability testing - just like any other method - overlooks some problems, even critical ones.

·      Expert reviews with highly experienced practitioners can be quite valuable – and, according to this study, comparable to usability tests in the pattern of problems identified – despite their negative reputation.

·      Focus on productivity instead of quantity. In other words, spend your limited evaluation resources wisely. Many of the teams obtained results that could effectively drive an iterative process in less than 25 person-hours. Teams A and L used 18 and 21 hours, respectively, to find more than half of the key problem issues, but with limited reporting requirements. Teams that used five to ten times as many resources did better, but the additional results in no way justified the considerable extra resources. This, of course, depends on the type of product investigated. For a medical device, for example, the additional resources might be justified.

Read more in our “Comparative Usability Evaluation (CUE-4)” paper. You can find more information about the paper in the left column.

Home    Free Resources    CUE    Products    Events    Books    About DialogDesign