Dialog Design DialogDesign ved Rolf Molich English English Dansk Dansk
DialogDesign, Skovkrogen 3, 3660 Stenløse, Denmark, dialogdesign@dialogdesign.dk, +45 4717 1731
Free Resources CUE Products Events Books About DialogDesign
CUE Papers CUE-1 CUE-2 CUE-3 CUE-4 CUE-5 CUE-6 CUE-7

CUE Paper Summary

Rolf Molich and Joseph S. Dumas,
“Comparative Usability Evaluation (CUE-4),”
Behaviour & Information Technology, Vol. 27, issue 3, 2008.



This paper reports on the approach and main results of CUE-4, the fourth in a series of Comparative Usability Evaluation studies. Seventeen experienced professional teams independently evaluated the usability of the website for the Hotel Pennsylvania in New York. Nine teams used usability testing while eight teams used expert reviews.

The CUE-4 results document a wide difference in resources applied and issues reported. The teams reported 340 different usability issues. Only nine of these issues were reported by more than half of the teams, while 205 issues (60%) were uniquely reported, that is, no two teams reported the same issue. Sixty-one of the 205 uniquely reported issues were classified as serious or critical problems.

The study also shows that there was no practical difference between the results obtained from usability testing and expert reviews for the issues identified. It was not possible to prove the existence of either missed problems or false alarms in expert reviews.

The paper further discusses quality measures for usability evaluation productivity.



Back to the CUE papers page

About this paper

The paper is 22 pages long in PDF format. The file size of the electronic version is 399 KB.

The paper was accepted by John Karat - one of the editors of BIT - on June 7, 2006.

Questions? Comments? Requests for e-prints?
Write to

Home    Free Resources    CUE    Products    Events    Books    About DialogDesign