TOURISM AS A METAPHOR FOR TEST DATA SELECTION

9.5 TOURISM AS A METAPHOR FOR TEST DATA SELECTION

In his book titled Exploratory Software Testing, Jim Whittaker, test engineering direc- tor at Google, discusses strategies for functional test data selection that equate testing a software product to exploring its various recesses, nooks and crannies so as to expose all its faults. Because these strategies are widely applicable, and cut across most exist- ing techniques, we briefly present them in this section. The software product is mapped into districts (just like a city), and the tours are partitioned by district (just like city tours). Specifically, six districts are identified:

1. The business district, which is the code that performs the core functionality of the product.

2. The historic district, which is the legacy code on which the application may have been built.

3. The tourist district, which is the code that delivers elementary functionality for novice system users.

4. The entertainment district, which is the code that delivers supportive features of the product.

5. The hotel district, which is the code that is active when the software is at rest.

6. The seedy district, which is the code that few users ever activate, but that may contain product vulnerabilities.

Many tours are scheduled for each one of these districts; we cite a few, referring the interested reader to the original source.

• Tours of the Business District ◦ The guidebook tour: This tour advocates reading the user manual in detail and

exercising the product’s functionalities according to the manual’s guidelines. This tour tests the system’s ability to deliver its advertised function, as well as the user manual’s precision in describing the system’s function.

◦ The skeptical customer tour: This tour advocates running the software product through a demo used by salespeople, but constantly interrupting the sequence

9.5 TOURISM AS A METAPHOR FOR TEST DATA SELECTION 189

of the demo to try variations that an end-user may be interested in, such as the following: What if I wanted to do this? How would I do that?

• Tours through the Historical District ◦ The bad neighborhood tour: This tour advocates running the software product

in such a way as to exercise parts of its codes that are likely to have faults; the assumption is that faults tend to congregate in a software product, and the more faults one finds in a part of the product, the more likely other faults are to be found therein.

◦ The museum tour: This tour advocates running the software product in such a way as to exercise parts of it that stems from legacy code; the rationale of this

tour is to expose any faults that may exist in the interface between legacy code and new code.

• Tours through the Entertainment District ◦ The supporting actor tour: This tour advocates exercising features of the soft-

ware product that share the screen with core features that most users typically use; they may be less visited than typical functions, but equally likely to have faults.

◦ The back alley tour: This tour advocates exercising features that are at the low end of the feature table of the software product, on the grounds that they may

also correspond to the least covered code of the product. • Tours through the Tourist District

◦ The collector’s tour: This tour advocates running the application on a suffi- ciently broad set of inputs and under a sufficiently diverse set of circumstances

that you can generate all the possible outcomes that the product is designed to deliver.

◦ The supermodel tour. This tour advocates to test the application, not on the basis of its functionality, but on the basis of its appearance.

• Tours through the Hotel District ◦ The rained out tour: This tour advocates initiating actions by the software

product and then canceling them as the earliest convenience offered by the application, to check whether the product resets its state properly and diligently.

◦ The couch potato tour: This tour advocates running the software product in such a way as to provide as little data as possible, to check the product’s ability

to proceed with incomplete information or by using default values. • Tours through the Seedy District

◦ The saboteur tour: This tour advocates initiating operations by the software product and then interfering with the proper execution of these operations

(e.g., launching an I/O operation and disconnecting the relevant device); the purpose of this tour is to test the product’s ability to handle exceptions.

◦ The obsessive compulsive tour: This tour advocates initiating the same opera- tion with the same input data over and over again and observing whether the

190 FUNCTIONAL CRITERIA

behavior of the system (in terms of whether the successive operations are inter- preted as multiple queries, whether they are properly separated from each other, whether they override each other, etc.) is consistent with its requirements.