INFO3315Human-Computer Interaction

    07-04 · Week 7

    Inspections: heuristic evaluation and cognitive walkthrough

    Expert inspection methods without users: Nielsen's ten heuristics, how many evaluators, the heuristic evaluation process, and the three cognitive walkthrough questions with the train ticket example.

    • Name and recognise all ten of Nielsen's usability heuristics.
    • Describe how a heuristic evaluation is run and how many evaluators are suggested.
    • Apply the cognitive walkthrough questions to a task step.
    • Contrast heuristic evaluation with cognitive walkthrough.

    22 min readJump to the quiz ↓

    Intuition

    You can find many usability problems before any user shows up. Inspections put experts in front of the design and ask them to look for trouble, either against a checklist of principles or by acting out a task as a newcomer would.

    Inspections use experts instead of users

    Experts use their knowledge of users and technology to review software usability. Critiques can be formal or informal. Two forms matter here. Heuristic evaluation is a review guided by a set of heuristics. Walkthroughs step through a pre-planned scenario noting potential problems.

    Heuristic evaluation judges the interface against ten principles

    A small set of evaluators examine the interface and judge its compliance with recognised usability principles. Nielsen’s original heuristics were derived empirically from an analysis of 249 usability problems. The lecture links a video rather than listing them, so the ten below, and the questions that go with them, come from the Week 8 worksheet.

    Exam detail

    1. Visibility of system status: does the design clearly communicate its state? Is feedback quick?
    2. Match between system and the real world: familiar terminology, real-world conventions.
    3. User control and freedom: go back a step, discoverable exits, cancel, undo and redo.
    4. Consistency and standards: industry conventions, consistent visual treatment.
    5. Error prevention: helpful constraints against slips, warnings before risky actions.
    6. Recognition rather than recall: keep important information visible, in-context help.
    7. Flexibility and efficiency of use: accelerators such as shortcuts and touch gestures, personalisation.
    8. Aesthetic and minimalist design: focus on essentials, remove distracting elements.
    9. Help users recognise, diagnose and recover from errors: clear error visuals such as bold red text, a solution that fixes the error immediately.
    10. Help and documentation: searchable help, provided in context at the moment of need.

    Nielsen suggests that on average five evaluators identify 75 to 80 percent of usability problems (Nielsen and Mack, 1994). The choice of heuristics depends on the evaluation goals. Guidelines can serve as heuristics for websites, and governments and large corporations must make websites accessible by law, so the Web Content Accessibility Guidelines (WCAG) are used.

    The procedure: a briefing to tell experts what to do; an evaluation period of 1 to 2 hours in which each expert works separately, taking one pass to get a feel for the product and a second to focus on specific features; then a debriefing where experts work together to prioritise problems. The lecture also shows severity ratings on a slide, but the scale did not extract, so check the slide.

    Benefits: few ethical and practical issues because users are not involved, and the best experts know the application domain and users. Problems: experts can be difficult and expensive to find, many trivial problems and false alarms are identified, and experts have biases.

    Cognitive walkthrough asks three questions about every step

    The cognitive walkthrough focuses on ease of learning. The designer presents an aspect of the design and usage scenarios. Experts are told the assumptions about the user population, context of use and task details, then walk through the prototype with the scenario. It is a systematic way to think like the user, drawing on Norman’s gulfs and a model of exploratory learning.

    The user has a task. 0. Will the user understand this sub-task is needed? The user explores the system for useful elements. 1. Will the correct action be obvious, for example a visible button? 2. Will the user understand the instructions, for example the label on the button? The user selects one to try and interprets the response. 3. Will the user interpret the machine action correctly?

    Compare

    Heuristic evaluation

    Evaluators review the whole interface against a set of heuristics (Nielsen’s ten). Each expert works alone, then they merge findings. Can find many trivial problems and false alarms.

    Cognitive walkthrough

    Experts walk a specific task and scenario step by step, asking the three questions. Focus is ease of learning. Considers the user’s task. Labour intensive.

    Try it: worked example

    Task: buy a one-way ticket to Town Hall on a cash-operated machine. Method 1 has the user click the destination then the journey type. The fare shows only after both clicks. Apply the three questions.

    Show answer and steps

    AnswerQ3 fails: there is no feedback until both selections are made. Fix: feedback on each select, indicate that steps 1 and 2 can be done in either order, and place the fare display after them.

    1. Goal breakdown from the designer: determine the fare to pay (indicate destination, indicate one-way or return), then dispense the ticket (pay money, get ticket and change).
    2. Ask the three questions for each action: is the correct action obvious, will the user understand the instructions, will the user interpret the machine action correctly.
    3. The lecture records the problem for Method 1 as: the user has to click both destination and journey type before there is any feedback. That undermines question 3, because nothing visible tells the user what their first click did.
    4. Revise the prototype: indicate that extra information is needed, show steps 1 and 2 can be done in either order, give feedback on each select, and reorganise so steps 1 and 2 are followed by the fare display.
    5. Repeat the walkthrough. Now the buttons light up and a new state appears, so all three answers are yes.
    6. Payment step: Q1 yes, Q2 yes, Q3 unclear because the user gets no feedback on money accepted so far. Alter the prototype to show how much has been paid, and Q3 becomes yes.

    The lecture leaves the case where the user has insufficient money as an in-class activity with no answer, so it is not solved here. Benefits of the walkthrough: done without users, considers the user’s task, quick and inexpensive. Problems: limited by the evaluator’s skills, and labour intensive because answering and discussing questions may take a long time.

    Pitfall

    The walkthrough questions are about a user learning by exploring one task. Heuristic evaluation asks about the whole interface against principles. Swapping the two is the standard distractor.

    Optional aside

    The Week 8 worksheet has you run a cognitive walkthrough in pairs on your own prototype (at least two tasks, one researcher leading, others noting problems), revise, then run a heuristic evaluation with another group using these ten heuristics and the Nielsen Norman Group template. That is project work, so only the method is covered here.

    Recall

    Which heuristic mentions undo and redo, and which mentions accelerators such as shortcuts?

    User control and freedom (heuristic 3) mentions undo and redo. Flexibility and efficiency of use (heuristic 7) mentions accelerators.

    Check yourself

    Select all that apply

    Source

    Lecture 7 PDF