Test, Test, and Retest 787
Step 14: Test, Test, and Retest 787
— Low cost. — Can be used to evaluate alternate solutions. — Can be performed by developers. — More structured than a heuristic evaluation. — Useful for assessing exploratory learning.
Disadvantages: — Tedious to perform. — May miss inconsistencies and general and recurring problems.
Guidelines: — Needed to conduct the walk-through are
• A general description of proposed system users and what relevant knowledge
they possess. • A specific description of one or more core or representative tasks to be per-
formed. • A list of the correct actions required to complete each of the tasks. — Review: • Several core or representative tasks across a range of functions. • Proposed tasks of particular concern.
— Developers must be assigned roles of • Scribe to record results of the action. • Facilitator to keep the evaluation moving.
— Start with simple tasks. — Don’t get bogged down demanding solutions. — Limit session to 60 to 90 minutes.
Description . In a cognitive walk-through, developers or users walk through an inter-
face in the context of representative user tasks. Individual task actions are exam- ined and the evaluators try to establish a logical reason why the user would perform each examined action. Actions are compared to the user’s goals and knowledge. Discrepancies and problems are noted and analyzed and the tasks modified as necessary. Walk-throughs require that the task definition methodol- ogy must have been properly accomplished in the system requirements stage. The user’s goals and assumptions must also be clearly defined before the walk- through is performed.
Advantages . Walk-throughs permit a clear evaluation of the task flow early in the
design process, before empirical user testing is possible. The earlier a design flaw can be detected, the easier it is to fix it. They can also be used to evaluate alternate design solutions. Walk-throughs are of low cost and can be performed by devel- opers. They are also more structured than a heuristic evaluation, being less likely to suffer from subjectivity because of the emphasis on user tasks. Walk-throughs are very useful for assessing exploratory learning, first-time use of a system with- out formal training.
Disadvantages . A cognitive walk-through is tedious to perform, and it may miss
inconsistencies and general and recurring problems. Studies have found that cognitive walk-throughs appear to detect far more problems than actually exist,
788 Part 2: The User Interface Design Process
compared to performance usability testing results (Koyani et al., 2004). In these studies only about 25 percent of predicted problems turned to be actual problems in a usability test. Thirteen percent of problems were missed in the walk-through. Like heuristic evaluations, walk-throughs appear to be most useful in identifying issues to evaluate in a usability test.
Guidelines . Needed to conduct the walk-through are a general description of pro- posed system users and what relevant knowledge they possess, a specific description of one or more core or representative tasks to be performed, and a listing of the correct actions required to complete each of the tasks. The walk- through should review several core or representative tasks across a range of func- tions, as well as proposed tasks of particular concern to the developers. Start with simple tasks to get the brain juices flowing. Don’t get bogged down looking for solutions to problems. Avoid detailed screen designs; they often inhibit assess- ment of the big picture. Limit a session to 60 to 90 minutes.