TEST DATA GENERATION FROM TABULAR EXPRESSIONS

9.2 TEST DATA GENERATION FROM TABULAR EXPRESSIONS

In the previous section, we have analyzed a criterion for test data selection that applies to deterministic specifications and partitions the domain of the specifica- tion (say F) using the nucleus of F as the equivalence relation EQ = s,s Fs=Fs . In this section, we see instances where the nucleus of F is either too coarse-grained or too fine-rained for our purposes, and we choose a different equivalence relation.

Tabular specifications are a form of formal specifications where complex functions that take different expressions according to many parameters can be represented in a way that highlights their dependencies and facilitates their analysis and their under- standing. As an example, consider a table that specifies the tax rates of individual tax- payers in a particular jurisdiction, as a function of their income, their marital status, and their number of dependents.

172 FUNCTIONAL CRITERIA

Function: Tax(X, d, t) t :

d : X : Marital

Number of Income bracket (in $K) status

dependents

150 < X ≤ 250 250 < X Single

0.20 × (X − 150) × (X − 250)

0.19 × (X − 150) × (X − 250) 3 or more

0.18 × (X − 150) × (X − 250)

17.6 + 36.6 + 0.24 filing

Married/ 1 0.07 × X

0.19 × (X − 150) × (X − 250) singly

0.18 × (X − 150) × (X − 250)

0.17 × (X − 150) × (X − 250) 4 or more

0.16 × (X − 150) × (X − 250)

We let X, d, and t be variables that represent, respectively, the taxpayer’s income, his/her number of dependents, and his/her marital status (filing status). The input space of this specification is defined by the set of values that these three variables take, where X is a real number, d is an integer, and t is a binary value (sin- gle, married). If we assume, for the sake of argument, that incomes of interest range between 0 and 1000,000, and that taxes are rounded to the nearest dollar figure, then the output of this function ranges between 0 and 186,600. If we were to apply the criterion of domain partitioning strictly, we would find that this function partitions its domain into 186,601 equivalence classes, hence T would have to have that many elements.

Yet, without knowing how candidate programs compute this function, we can assume with some level of confidence that each entry in this tabular expression cor- responds to a distinct execution path; hence if we generated one test datum for each entry rather than one test datum for each tax value, we would get a much smaller test data set (35 elements rather than 186,601) without perhaps much loss of effectiveness. To this effect, we apply domain partitioning to a function Tax derived from Tax by replacing each expression in the tabular representation of Tax by a (distinct) constant. The rationale for this substitution is the assumption that if and only if an expression in the table of Tax produces a correct value for one element within its domain of appli- cation, then it produces a correct value for all elements within its domain of applica- tion. This yields the following function.

9.2 TEST DATA GENERATION FROM TABULAR EXPRESSIONS 173

Function: Tax '(X, d, t) t :

Marital status Number of

Income bracket (in $K)

dependents

60 < X ≤ 150 150 < X ≤ 250 250 < X t = Single

X≤ 20 20 < X ≤ 60

d= 1 C 1,1,1

C 1,1,2

C 1,1,3

C 1,1,4 C 1,1,5

C 1,2,4 C 1,2,5

C 1,3,4 C 1,3,5

t = Married

d= 1 C 2,1,1

C 2,1,2

C 2,1,3

C 2,1,4 C 2,1,5

d= 2 C 2,2,1

C 2,2,2

C 2,2,3

C 2,2,4 C 2,2,5

C 2,3,4 C 2,3,5

C 2,4,4 C 2,4,5

Applying space partitioning to function Tax using the nucleus of this function, we find the following test data.

Marital Number of Income bracket (X, in $K) status/

dependents Taxpayer

150 < X ≤ 250 250 < X status

X≤ 20 20 < X ≤ 60

60 < X ≤ 150

t = Single

(200, 1, S) (300, 1, S) d =2

d= 1 (10, 1, S)

(40, 1, S)

(100, 1, S)

(200, 2, S) (300, 2, S) d≥3

(200, 4, S) (300, 4, S) t = Married

d= 1 (10, 1, M)

(40, 1, M)

(100, 1, M)

(200, 1, M) (300, 1, M)

(200, 2, M) (300, 2, M) d =3

d= 2 (10, 2, M)

(40, 2, M)

(100, 2, M)

(200, 3, M) (300, 3, M) d≥4

(200, 5, M) (300, 5, M)

To check that the boundaries between the various income brackets are pro- cessed properly by candidate programs, we may also want to duplicate this table for all boundary values of X, namely X=20, 60, 150, 250, and 1000 (assuming that 1M is the maximum income under consideration). This yields the following test data.

174 FUNCTIONAL CRITERIA

Marital Number of

Income bracket (X, in $K)

status/ dependents Taxpayer

X≤ 20 20 < X ≤ 60 60 < X ≤ 150 150 < X ≤ 250 250 < X status

t = Single

d= 1 (10, 1, S)

(40, 1, S)

(100, 1, S)

(200, 1, S) (300, 1, S)

(250, 1, S) (1000, 1, S) d =2

(200, 2, S) (300, 2, S)

(250, 2, S) (1000, 2, S) d≥3

(200, 4, S) (300, 4, S)

(250, 4, S) (1000, 4, S) t = Married

d= 1 (10, 1, M) (40, 1, M) (100, 1, M) (200, 1, M) (300, 1, M) (20, 1, M) (60, 1, M) (150, 1, M) (250, 1, M)

(1000, 1, M) d= 2 (10, 2, M) (40, 2, M) (100, 2, M) (200, 2, M)

(300, 2, M) (20, 2, M) (60, 2, M) (150, 2, M) (250, 2, M)

(1000, 2, M) d =3

(10, 3, M) (40, 3, M) (100, 3, M) (200, 3, M) (300, 3, M) (20, 3, M) (60, 3, M) (150, 3, M) (250, 3, M)

(1000, 3, M) d≥4

(10, 5, M) (40, 5, M) (100, 5, M) (200, 5, M) (300, 5, M) (20, 5, M) (60, 5, M) (150, 5, M) (250, 5, M)

(1000, 5, M)

Assuming that the specification is valid (is not in question) and a candidate pro- gram p computes taxes as a linear function of income within each partition of the input space, it is highly unlikely that program p runs successfully on all the test data pre- sented above, yet fail on any other valid input.

We consider a second example, where we show that the important criterion for space partitioning is not the nucleus of the function, but rather how the specifica- tion is represented in tabular form. We consider a (fictitious) tabular specification that represents the graduate admissions criteria at a university, depending on stand- ard graduate record examination (GRE) scores for quantitative reasoning (G) and grade point average (GPA) (A); also the admissions committee lends different levels of credibility to different institutions and hence interprets the GPA with dif- ferent levels of confidence, according to whether the candidate did his undergrad- uate degree in the same institution, in another North American institution, or elsewhere, as shown by the table below. We assume that the specification provides three distinct outcomes and they are admission (Ad), rejection (Re), and condi- tional acceptance (Cond), where the latter outcome places the application on hold until an admitted student declines the admission, thereby freeing up a spot for admission.

9.2 TEST DATA GENERATION FROM TABULAR EXPRESSIONS 175

Admission criteria Institution, I

GPA, A

GRE score, G

160 ≤ G < 165 165 ≤ G Same

Institution 2.3 ≤ A < 2.9

Cond Cond

Another North 2.9 ≤ A

Cond

Ad Ad Ad

American Institution

Cond Cond

Overseas 2.9 ≤ A

Cond Ad

Re Cond

The input space for this function is the Cartesian product of three variables, namely

G, A, and I. If we apply space partitioning using the nucleus of this function, we find three equivalence classes, corresponding to the three possible outputs. But in practice, any candidate program most likely proceeds by combining the conditions shown above; hence it makes more sense to partition the input space by combining these conditions, which gives 36 classes rather than 3. If for each class we select a test datum in the middle of the interval and at the boundary, we find the following test data set.

Admission criteria Institution, I

GPA, A

GRE score, G

155 ≤ G < 160 160 ≤ G < 165 165 ≤ G Same

(162,S,3.5) (168,S,3.5)

(160,S,2.9) (165,S,2.9)

(162,S,2.5) (168,S,2.5)

(0,S,2.3)

(155,S,2.3)

(160,S,2.3) (165,S,2.3)

(162,S,2.0) (168,S,2.0 )

(0,S,.0)

(155,S,.0)

(160,S,.0) (165,S,.0)

176 FUNCTIONAL CRITERIA

Another 2.9 ≤ A

(120,N,3.5)

(158,N,3.5)

(162,N,3.5) (168,N,3.5)

(160,N,2.9) (165,N,2.9)

American Institution

(162,N,2.5) (168,N,2.5)

(160,N,2.9) (165,N,2.9)

(162,N,2.0) (168,N,2.0)

(0,N,.0)

(155,N,.0)

(160,N,.0) (165,N,.0)

Overseas 2.9 ≤ A

(120,O,3.5)

(158,O,3.5)

(162,O,3.5) (168,O,3.5)

(160,O,2.9) (165,O,2.9)

(162,O,2.5) (168,O,2.5)

(0,O,2.9)

(155,O,2.9)

(160,O,2.9) (165,O,2.9)

(162,O,2.0) (168,O,2.0)

(0,O,.0)

(155,O,.0)

(160,O,.0) (165,O,.0)

For completeness, we could enrich the data set by combining normal values with boundary values.