Dialog Design DialogDesign ved Rolf Molich English English Dansk Dansk
DialogDesign, Skovkrogen 3, 3660 Stenløse, Denmark, dialogdesign@dialogdesign.dk, +45 4717 1731
Home Free Resources CUE Products Events Books About DialogDesign
CUE Papers CUE-1 CUE-2 CUE-3 CUE-4 CUE-5 CUE-6 CUE-7 CUE-8

CUE-8 - Task Measurement

You can’t manage what you can’t measure - but can you manage measurements to generate reproducible and useful results?

CUE-8 compared practical approaches to usability task measurement. Fifteen participating teams carried out independent, simultaneous measurements of the Budget.com car rental website and compared results.

CUE-8 results were presented and discussed at a workshop at the UPA 2009 conference in Portland, OR, USA, on Tuesday 9 June 2009.

The teams are currently preparing several papers describing the CUE-8 results.  More details will appear here before the end of 2009.

Available Downloads

·       CUE-8 instructions to participating teams (PDF, 40 KB).

·       All 15 reports and appendices plus workshop agenda plus results overview (Zip, 17 MB, password cueeight)

 

Preliminary advice from CUE-8 about Task Measurement

·      Adhere strictly to precisely defined measurement procedures to get reproducible results. Quantitative usability measurements are much less tolerant than traditional qualitative usability tests in this respect.

·      Use measurement methods that have been well researched and standardised.

·      Report time on task, success/failure rate and satisfaction.

·      Never include the time for failed tasks in time on task.

·      Understand the inherent variability from samples and provide confidence intervals around your results. Keep in mind that with regard to time on task, we do not have an adequate quantitative model of how to describe our data.

·      Justify the composition and size of your participant samples. This is the only way you have to allow your client to judge how much confidence they should place in your results.

·      When using unmoderated methodologies ensure that some kind of triangulation keeps your data credible. Do not report unconfirmed data which does not look credible.

Home    Free Resources    CUE    Products    Events    Books    About DialogDesign