Assessment of the quality of usability tests

The CUE-studies show that there is a wide variety of approaches to usability testing. Not all these approaches are optimal.

Sub-optimal usability tests may lead to overlooking serious and critical usability problems. They can also lead to spending resources on the correction of incorrect or insignificant usability problems.

DialogDesign recommends the following approaches to ensure the quality of usability tests:

  • Usability test training and certification, CPUX-UT.
    This includes an individual assessment of a simple usability test carried out by the candidate
  • Remote individual assessment of usability test
    within your organisation. This includes an inspection of the usability test script, videos of usability test sessions and usability test reports.
  • On-site individual assessment of usability test
    within your organisation. Rolf Molich observes and gives feedback on your usability tests.

All assessments are based on a carefully reviewed, publicly available checklist.

Seven tips for a better usability test

The most important advice that the evaluators currently have from their assessments of practical usability test submissions from CPUX-UT candidates are:

  • Audio quality
    It must be easy to hear what the test participant or the moderator are saying.
    Tips: Use a separate microphone for the recording. Do not use the built-in microphone on a laptop. Check the quality of your audio by making a test recording. Also, check the audio quality after the first usability test session.
  • Video quality
    It must be easy to read the contents of the screen on the video.
    Tips: Check the quality of your videos by making a test recording. Also, check the video quality after the first usability test session.
  • Time management
    Manage the limited time you have with the usability test participant in the usability test session well
    Tips: Stop test participants if they explore beyond the given structured tasks. Proceed to the next task when the usability problem is clear
  • Test tasks
    Write test tasks that match your usability test plan.
    A usability professional must be able to follow precise instructions, such as those provided in the usability test plan exactly.
    Avoid usability test tasks are trivial or have hazy goals.
  • Curiosity
    Moderators should follow up on important, interesting issues that were raised in a usability test session, either after the task has been completed or during the debriefing.
  • Say as little as possible
    Tips: Don’t ask questions about issues that can be clearly observed. Don’t turn the usability test session into an interview, for example by repeatedly asking “What would you do?”
  • Usability test report
    Only describe findings that are evident from what happened in the usability test sessions. Avoide findings that are based on inspections or test participants’ opinions
    Describe all important findings from the usability test sessions