Web Usability Engineering Methods

11.5 Web Usability Engineering Methods

Usability engineering methods can be integrated in arbitrary development process models. This section is based on requirements analysis, design, implementation, and operational phases to illustrate the phase-dependent use of different usability engineering methods. All considerations should be such that they can be transposed to realistic process models (see Chapter 10) without major difficulties. It is important that the carrier process supports an iterative approach, and that these iterations are also (and particularly) controlled by the degree to which usability goals are met (see section 11.5.1). Figure 11-6 shows this type of iteration control, using the usability engineering process suggested by Mayhew (1999) as an example.

To do justice to the Web’s peculiarities discussed in section 11.3, we can set specific focal points along the spectrum of methods for usability engineering. The most important aspect is the reduced availability of users, so that user-based methods are applicable only to a limited extent, as shown in Table 11-2. One solution to compensate for this shortcoming is based on the usage- centered approach developed by Constantine and Lockwood (see section 2.5.1 and Constantine and Lockwood (2002) – the most important criteria of this concept are integrated in Table 11-2, and details will be discussed in section 11.5.1). However, this and similar approaches should not tempt us to totally give up on trying to involve users. As soon as a chance is seen to involve users, it should better be done earlier than later in Web usability engineering. In the process model discussed below, methods from one column in Table 11-2 can easily be allocated to the other column, though with lower priority. Web applications literally take a medium position between conventional applications at one extreme and information-centered Web sites with trivial interactivity at the other extreme. This essentially means that setting focal points depends on the characteristics of a Web application, and that methods from both of the other categories have to be applied.

11.5 Web Usability Engineering Methods 233

Figure 11-6 Iterative usability engineering (simplified from Mayhew 1999).

We will categorize Web applications into two classes for the further discussion of user participation: anonymous Web applications include information-centered Web applications and Web applications with unknown user target groups, while non-anonymous Web applications correspond to conventional software on a Web platform (normally in an intranet) with known user target groups.

The tasks involved in a usability engineering process can be roughly allocated to four major roles (the literature sometimes uses finer classifications). These four roles fall more or less into the “functional” category of Figure 9-3; they are ideally but not necessarily occupied by different persons:

• The systems analyst is in charge of the requirements analysis and the conceptual design. He or she analyzes the target group, their tasks to be supported by the system, and the context of the application under development.

• The interface designer is responsible for the fundamental design of the user interface and the application’s look and feel. • The usability expert checks and evaluates models, guidelines, and standards, conducts tests, analyzes their results, and issues change requests. • The implementer implements the system (or perhaps just the user interface in larger projects) and is knowledgeable in the field of interface design to be able to meet the guidelines set by the usability expert.

234 Usability of Web Applications Table 11-2 Setting focal points in usability engineering for Web applications

Phase

Focal Points

User-centered approach

Usage-centered approach

Archetype:

Archetype:

conventional application

information-centered Web application

Requirements

Focus on usage: meeting the analysis

Focus on users: user experience

and satisfaction. Methods:

tasks. Methods: competitive

meetings, interviews, focus

analysis, task analysis and

groups.

task modeling (scenario techniques).

Design and

Guided by models. Selective implementation

Guided by user requirements.

Extensive and generally

and normally indirect user

direct user participation

participation (explorative

(user analysis, participative

modeling, model validation,

design (paper mock-ups,

usability inspections, remote

card sorting), user feedback

usability tests).

analysis (focus groups), usability tests).

Operation

Training, evaluation of

Log file analysis, server

helpdesk logs.

statistics, user feedback analysis.

The following section briefly introduces the usability engineering activities, concentrating on the specifics in Web application development. Details of the general basics of usability engineering can be found in the standard literature, for example, Nielsen (1994), Nielsen (1999a), Mayhew (1999).

11.5.1 Requirements Analysis The general principles of requirements engineering for Web applications have been described in

Chapter 2. This section deals with those aspects of the requirements analysis that are of particular significance for usability engineering.

A requirements analysis begins with the formulation of a “vision”, describing the basic function and performance characteristics, essential use cases, and the envisioned target groups. This lays the foundation-stone for the task and user models to be created in the requirements analysis, and controls further development phases within usability engineering.

A competitive analysis (which is allocated to the usage-centered methods; see Table 11-2) is particularly significant for the usability of Web applications. The systems analyst rounds off the big picture of the Web application under development by studying other Web applications and the relevant literature. This is not aimed at copying an existing Web application, but considering best practices and negative examples versus our own purposes and requirements (e.g., corporate design) to work out a concept for the Web application we want to create.

One of the most important results of the analytical phase for successful usability engineering is a clear definition of qualitative and quantitative usability goals by the systems analyst or

11.5 Web Usability Engineering Methods 235 usability expert. These goals should be formulated so concretely that achieving the goals in

the course of the process can be checked effectively and used for iterative process control (Figure 11-2). In 1996, Siegel suggested to differentiate between three basic types of purposes for Web applications (Table 11-3), for which specific usability goals can be identified. The goals mentioned in the table are just examples but they illustrate the structured approach.

Table 11-3 Purposes of Web applications (Siegel 1996) and usability goals

Purpose of a

Examples of usability goals Web application

Information Focus on content,

60% of the test publications

To be the most popular

restaurant guide in

persons prefer it

the country

over the products X and Y/number of unsuccessful searches < 5%

Entertainment Marketing, typical

The “satisfaction” surfing

Highly attractive for

advertising

factor has to reach

customers, affiliate

the value X in the

companies/positive

test/has to be above

average Exchange

user attitude

E-commerce,

Make online table conventional

Most efficient table

reservation faster applications on Web

reservation method

than a phone call to platforms

in the competitive

field

the restaurant (less than 20 seconds)/ 10% faster than a competitive site

Another important usability-oriented activity of the systems analyst in preparing the require- ments analysis is normally the specification of user profiles in the sense of a user-centered design process (ISO/IEC 13407). The previously mentioned usage-centered approach suggests itself for Web applications in view of reduced user availability. To form a basis for the requirements analysis, the user categories are characterized not primarily by observing individual users, but by using task and usage models, deduced from the requirements inherent in the tasks to be supported.

The usage-centered design process requires a comprehensive task analysis, and priorities should be meaningfully set (Which tasks must be supported? Which tasks should be optionally supported?). Moreover, it is necessary to clarify whether the usability focus should be on ease of learning or ease of use, since achieving both of these goals concurrently is difficult, as novices and experts have different usage characteristics. An important distinguishing criterion is the expected frequency of use: booking a winter vacation on the Web site of a travel agency will most likely demand ease of learning, while weekly standard transactions in an online banking application demand ease of use.

236 Usability of Web Applications In the course of analyzing the tasks, we should also study the side conditions of using a Web

application, for example, the prevailing types of devices (PC, PDA, smart phone), or the users’ conditions (moving, sitting, relaxed, hectic). The use of scenario techniques is often meaningful to ensure that all important usability aspects (users, tasks and context) are taken into account.

A scenario collects the above environmental factors and checks their relevance for the usability of the system under development. A scenario-related design could, for example, consider the possibility that a Web application is used by a customer who has just seen a special offer in the window of a travel agency, and immediately uses his or her PDA to get more details. The limited interaction possibilities, suboptimal use conditions, and time restrictions in this scenario represent additional non-functional side conditions for the Web application.

Users can participate in the requirements analysis directly or indirectly. Indirect user partici- pation is easier to realize than a direct one for anonymous Web applications. This approach uses interview data or usage statistics from other organizations (e.g. GVU, 9 AIM 10 ), for example, to assess the need for the planned product, or the Web’s suitability for a sales department. Making room reservations, obtaining information about a vacation resort, and similar activities can be excellently supported by the Web. However, things are slightly more difficult if we want to market a gourmet restaurant on the Web because, in addition to the obligatory quality of the dishes and beverages offered, the atmosphere and the personal service for the guests play an important role. Though the above-mentioned methods focus on the users, users are not involved in the concrete case. This is the reason why Table 11-2 allocates them to the usage-centric area.

Direct user participation in a user-centric approach is based on focus groups or interviews, to mention two examples. Focus groups are group discussions focusing on specific issues, and it is important not to let the participants slip too much from the main topics. Focus groups serve to obtain a cross-section of opinion from the potential user group. However, there is a risk that user opinions tend towards a certain direction due to potential peer pressure. Interviews can be free or structured. In particular free interviews have the drawback that a large number of individual opinions have to be brought to a common denominator.

From the usability engineering standpoint, the results from the requirements analysis can be summarized as follows (see also Figure 11-2):

• User profiles (especially for non-anonymous Web applications). • Task analysis, scenarios, and use case model. • Specification of the platform properties (devices, browsers, plug-ins). • Qualitative and quantitative usability goals (see Table 11-3).

Mayhew (1999) defines an integrated result document that includes all these results. It is completed by a short summary of all those design principles that should find special consideration in a specific project. This document, known as a styleguide, provides the driving force for the design and should be continually enhanced in all subsequent development steps. Consequent observance of the styleguide by all people participating in the development project can help achieve maximum consistency for the Web application under development.

9 Visualization and Usability Center at the Georgia Tech University. 10 Austrian Internet Monitor.

11.5 Web Usability Engineering Methods 237

11.5.2 Design The interface designer takes the results from the requirements analysis and develops a conceptual

model of the user interface. This model defines the basic structure of the Web application, facilitating the developer to “see through” the application. Initial versions of this model are based on the important system subfunctions, perhaps those specified in core use cases. The model focuses on basic representation aspects, metaphors, and navigation and interaction principles. This conceptual model should be concretized on a case-by-case basis to better illustrate how it can be converted into the real thing.

The user participation methods (see Table 11-2; user-centric approach) that we can use in this phase include, for example, storyboarding, paper mock-ups, and card sorting, in focus groups with members that ideally correspond well to the target group. Similar to a comic strip, a storyboard symbolizes sequences and helps to identify potential stumbling-blocks. Paper mock-ups are more or less mature sketches of screen contents designed by the interface designer. For example, they can be presented to the participants in a focus group to have their quality evaluated, e.g., with regard to coloring and positioning of functional elements. Card sorting serves to find out how contents fit best into the structure of a Web application.

For example, assuming that accommodation and restaurant are two distinct areas in a hotel, room reservations and table reservations are handled differently. For a guest, however, both could fall under the “reservation” category. To find this out, we produce cards for all contents of the Web application under development and fill them with keywords, such as “reserve table”. Next, people are asked to bring these cards into a structure they think would be meaningful, by having them arrange the cards in different groups. The result supplies valuable indications as to how the Web application should be structured, e.g., for the contents of menus. In contrast to the presentation modeling described in section 3.7, the interface prototypes resulting from the methods described here can be very concrete and detailed, because the “consumers” of these models are average users rather than development team members.

Subsequently, the model is evaluated by the usability expert. Such a usability inspection (belonging to the spectrum of usage-centered methods; see Table 11-2) identifies potential weaknesses early in the design, e.g., inconsistent sequences, potentially excessive mental stress for users, confusing navigation structure, etc.

As soon as the conceptual model has stabilized (in the sense of an iterative approach) the interface designer and the implementer can jointly elaborate a detailed design of the user interface. For this task, it is recommended to fall back on existing design knowledge (see sections 11.4 and 11.6.1).

User participation in the design phase takes place via two channels: • Prototyping: In contrast to paper mock-ups, the prototypes used here exhibit rudimentary

functions, showing what the user interface of the finished system would look like. • Usability tests: The detailed design can be validated for usability in a user test, particularly if prototypes are available. These tests should come closest to a real context or involve real tasks. For Web applications with a known target group, a conventional usability test is feasible and meaningful. And remote usability tests can be used for Web applications with unknown user groups.

238 Usability of Web Applications The sequence of a remote usability test can be outlined as follows. Out of a number of users with

known profiles a random sample representative of the Web application’s target group is drawn. These users are given specific tasks, which have to be solved within the Web application. The degree of meeting these tasks, problems that may arise, the satisfaction of the users, and perhaps the time required to solve each of the tasks, are assessed from user feedback, and partly with appropriate technical solutions. In this context, it is beneficial to install special software on the registered user’s machine (to be able to log keyboard and mouse inputs), and additional video recording of the facial expressions of users during the test, e.g., Web cams, is possible. Another benefit of remote usability tests is that the external validity is better under real-world conditions (work environment and technical equipment, e.g., the Internet connection), compared with lab surveys. Moreover, the cost for the participants is reduced, because they don’t have to leave their usual place of work. Though users participate in remote usability tests, this method is allocated to the usage-centered approach in Table 11-2. The reason is that, in contrast to classic usability tests, the tester doesn’t dispose of a certain percentage of information, which is hard to calculate. This information would be available under lab conditions, for example, aspects of the behavior or verbal expressions of the test persons. The result is that more inductive conclusions have to be drawn and more thinking about the model is required, compared with a classical usability test.

The following results of the design process are used to correct and complete the implementation styleguide:

• Conceptual model • Detailed design • Test results in a reproducible documentation of (re)design decisions.

11.5.3 Implementation Apart from the implementer, the usability expert plays the most important usability engineering

role in the implementation phase. The usability expert’s quality assurance tasks consist in checking the consistency of the Web application (particularly if there are several implementers), the observance of guidelines and standards documented in the styleguide, and the development of strategies if requirements have changed.

Involving users in this phase can again be in the form of focus groups. It is important to invite the same people who had been involved in a preliminary phase and ask them whether or not the real-world system corresponds to what they had expected, or whether or not critical comments previously expressed have been considered satisfactorily.

From the usability engineering perspective, the documentation about the achievement of the usability goals defined in the requirements analysis is the most important result of the implementation phase.

11.5.4 Operation The goal of usability engineering during the operational period of an application is to derive

requirements for future versions from the users’ experience in handling the system. Based on the more intensive use of the system, compared with the development phases, long-term usage

11.6 Web Usability Engineering Trends 239 is a particularly rich source of collecting potential information about the Web application’s

usability. To this end, we identify offline methods with direct user contact, e.g., focus groups, which are conducted prior to version changes (see section 11.5.1), and online methods, such as log file analysis or collaborative filtering (see Chapter 8), allowing to observe and analyze user behavior over the Internet. However, these survey methods influence various aspects of one of the attributes to be measured, namely the usability criterion “satisfaction”, to a more or less large extent: privacy, system control, security, etc. Careless handling of these methods is, therefore, questionable for data privacy and ethic reasons and for theoretical measuring reasons.

A possibility to reduce these problems is to maintain transparency towards the users, who should

be informed about the methods and give their consent. The analysis during operation results in specifications of change requests and styleguide extensions, thus preparing the next iteration through the product lifecycle.