Test case sources

10.2.2 Test case sources

There are two basic sources for test cases:

Random samples of real life cases. Examples: – A sample of urban households (to test a new municipal tax informa-

tion system) – A sample of shipping bills (to test new billing software) – A sample of control records (to test new software for control of man-

ufacturing plant production) – A recorded sample of events that will be “run” as a test case (to test

online applications for an Internet site, and for real-time applications).

Synthetic test cases (also called “simulated test cases”) prepared by test designers. This type of test case does not refer to an existing customer, shipment or product but to combinations of the system’s operating

234 conditions and parameters (defined by a set of input data). These combi- nations are designed to cover all known software operating situations or

10 at least all situations that are expected to be in frequent use or that belong Sof

to a high error probability class. For the equivalence class method, see Section 9.5.1.

tw are tes

The implications of using each test case source are summarized and com- pared in Table 10.3.

ting – impl In most cases, the test case file preferred should combine sample cases

with synthetic cases so as to overcome the disadvantages of a single source of test cases and to increase the efficiency of the testing process. In the case of combined test case files, test plans are often carried out in two stages: in the first stage, synthetic test cases are used. After correction of the detected

ement

errors, a random sample of test cases is used in the second stage.

ation

Table 10.3: Comparison of test data sources Implication

Type of test case source

Synthetic test cases Effort required to

Random sample of cases

High effort; the parameters of prepare a test

Low effort, especially where

expected results are available and each test case must be case file

need not be calculated

determined and expected results calculated

Required size of Relatively high as most cases refer Relatively small as it may be test case file

to simple situations that repeat

possible to avoid repetitions of

themselves frequently. In order to

any given combination of

obtain a sufficient number of

parameters

non-standard situations, a relatively large test case file needs to be compiled

Low efforts (high efficiency) due perform the

Efforts required to

High efforts (low efficiency) as

tests must be carried out for large to the relatively small test case software tests

test case files. The low efficiency

file compiled so as to avoid

stems from the repetitiveness of

repetitions

case conditions, especially for the simple situations typical to most real-life case files

Effectiveness –

Relatively low – unless the test ■ Relatively high due to good probability of error

coverage by design detection

case files are very large – due

to the low percentage of

Good coverage of erroneous

uncommon combinations

situations by test case file

of parameters

design

No coverage of erroneous

Little possibility of identifying

situations

unexpected errors as all test

Some ability to identify

cases are designed according

unexpected errors for unlisted

to predefined parameters

situations

Stratified Sampling 235 Substantial improvement in the efficiency of random sampling of test cases is achieved by using a stratified sampling procedure rather than standard ran-

10.3 Autom

dom sampling of the entire population. Stratified sampling allows us to break down the random sample into sub-populations of test cases, thereby reducing the proportion of the majority “regular” population tested while increasing the sampling proportion of small populations and high potential error popu- lations. This method application minimizes the number of repetitions at the

ated tes

same time that it improves coverage of less frequent and rare conditions. As an example, Garden City’s population of about 100 000 households is divided between the city itself (70%), suburb Orange (20%), suburb

ting

Lemon (7%) and suburb Apple (3%). The suburbs and the city differ sub- stantially in the characteristics of their housing and socio-economic status. Some 5% of the households, the great majority of them city dwellers, enjoy tax reductions entailing 40 different types of discounts (disabled, very large families, low-income single-parent families with more than six children, etc.). Originally, the standard 0.5% sample had been planned. It was later replaced by the following stratified random sample:

Households Standard sampling Stratified sampling

(no.) Regular households

(no.)

(no.)

100 Households enjoying discounts

25 250 Suburb A

50 Suburb B

35 50 Suburb C

Test cases for reused software. It is quite common for reused software to include many applications not required for the current software system in addition to the required applica- tions. In situations of this kind, planners should consider which reused software modules should be tested. Other modules of the reused software will not be tested.