The purpose of CUE-8 was to compare practical approaches to usability task measurement.
Fifteen participating teams carried out independent, simultaneous measurements of the Budget.com car rental website and compared results.
CUE-8 results were presented and discussed at a workshop at the UPA 2009 conference in Portland, OR, USA, on 9 June 2009.
Practitioner’s Take Away
- Adhere strictly to precisely defined measurement procedures for quantitative tests.
- Report time-on-task, success/failure rate and satisfaction for quantitative tests.
- Exclude failed times from average task completion times.
- Understand the inherent variability from samples. Use strict participant screening criteria. Provide confidence intervals around your results if this is possible. Keep in mind that time-on-task is not normally distributed and therefore confidence intervals as commonly computed on raw scores may be misleading.
- Combine qualitative and quantitative findings in your report. Present what happened (quantitative data) and support it with why it happened (qualitative data). Qualitative data provide considerable insight regarding the serious obstacles that users faced and it is counterproductive not to report this insight.
- Justify the composition and size of your participant samples. This is the only way you have to allow your client to judge how much confidence they should place in your results.
Papers About CUE-8
Rent a Car in Just 0, 60, 240 or 1,217 Seconds? – Comparative Usability Measurement, CUE-8
by Rolf Molich, Jarinee Chattratichart, Veronica D Hinkle, Janne Jul Jensen, Jurek Kirakowski, Jeff Sauro, Tomer Sharon, Brian Traynor
JUS, the Journal of Usability Studies, November 2010
Rent a car in just 60, 120, 240 seconds – Comparative Usability Measurement
by Rolf Molich, Jurek Kirakowski and Tomer Sharon
90-minute session at the UPA 2010 conference in Munich, Germany, May 2010