Here are the differences at a glance:
| Common heuristic evaluation | Orthodox heuristic evaluation© | Usability inspection | |
|---|---|---|---|
| Are heuristics used? | Yes | Yes | Yes |
| Is experience from usability tests and the literature used? | In principle, no In practice, yes | Definitely no | Yes |
| When is a heuristic associated with the usability problem? | After the problem has been found | When the problem is found | Not applicable |
| Can users be involved as evaluators? | Yes | Yes | Yes |
| Can users be involved as usability test participants? | No | No | No |
| How are the heuristics used? | Heuristics are assigned to usability findings as an afterthought. | The heuristics drive the evaluation | Heuristics are used on par with dialogue principles, user interface guidelines and user requirements |
| Does the name of the method sound cool? | Yes | Definitely yes | No |
| Widely used? | Yes | Definitely no | Yes |
The following detailed definitions may help you understand the differences better
Usability evaluation: A process through which information about the usability of an interactive system is gathered in order to improve the interactive system.
Usability inspection: A usability evaluation based on the judgment of one or more evaluators who examine or use an interactive system to identify potential usability problems and deviations from established dialogue principles, heuristics, user interface guidelines and user requirements.
Common heuristic evaluation: A usability inspection in which one or more evaluators compare an interactive system to a list of heuristics and identify where the interactive system does not follow those heuristics. After usability findings have been identified, each one is associated with one or more heuristics.
Orthodox heuristic evaluation©: A usability inspection in which one or more evaluators compare an interactive system to a list of heuristics and identify where the interactive system does not follow those heuristics. The heuristics must drive the evaluation; only findings that can be clearly associated with at least one heuristic may be reported.
The differences between common heuristic evaluation and orthodox heuristic evaluation© are shown in bold. In practice, most UX professionals do a usability inspection. When they say that they do a heuristic evaluation, they actually do a Common heuristic evaluation, which is very similar to a usability inspection where the heuristics are added as an afterthought.
For a description of the evaluation process, please see How to do a heuristic evaluation.
What the CUE-studies say about heuristic evaluation
The CUE-studies, in particular the CUE-4 study, showed that both common heuristic evaluation and usability inspection are efficient and reliable. Contrary to usability testing, they do not require users (usability test participants) and a technical setup.
The CUE-studies also showed that in practice most “heuristic evaluations” are common heuristic evaluations, that is, usability inspections. The CUE-studies also showed that the required association of a usability finding with one or more heuristics is quite often doubtful or even incorrect. Many usability findings cannot be easily associated with a heuristic. This means that they would not be reported in an orthodox heuristic evaluation©.
To make the difference between the two forms of heuristic evaluation clear, I coined the term “orthodox heuristic evaluation©,” which describes the original method. Hardly anyone uses orthodox heuristic evaluation© today. This is a pity because orthodox heuristic evaluation© prevents false positives by relying solely on usability knowledge that is widely used and has stood the test of time.
Source of definitions (except common and orthodox heuristic evaluation): CPUX Foundation level curriculum.