AN ILLUSTRATIVE EXAMPLE

14.7 AN ILLUSTRATIVE EXAMPLE

As further illustration, we consider the following sorting program and briefly compute its metrics as well as the non-determinacy of three possible sorting specifications that the program may be tested against.

int N=100; void sort ()

{int c, d, p, swap; c=0; while (c<N-1) {p=c; d=c+1; while (d<N) {if (a[p]>a[d]) {p=d;} d++;}

if (p!=c) {swap=a[c]; a[c]=a[p];

a[p]=swap;} c++;}}

14.7.1 Cyclomatic Complexity

If we draw the flowchart of this program, we find that it has 14 edges and 11 nodes. Hence its cyclomatic complexity is 14 − 11 + 2 = 5.

328 METRICS FOR SOFTWARE TESTING

14.7.2 Volume

If we analyze the source code of this program, we find that it has 108 lexemes taken from a vocabulary of 27 distinct lexemes. The volume of this program is:

108 × log 27 = 513 53

14.7.3 State Redundancy

If we assume that integer variables are represented by 32 bits, then the entropy of the state of this program is 104 × 32 bits = 3328 bits. Now, if we assume that the cells of the array (as well as variable swap) range between 1 and 400 and that index variables (c, d, p) range between 0 and 100, then the entropy of the random variable that repre- sents the initial data of the program is 101 × log(400) + 3 × log(100) = 893 bits. Hence the state redundancy of the initial state is 3328 − 893 bits = 2435 bits.

In order to compute the state redundancy of the final state of this program, we refer to a result to the effect that when an array of size N is sorted, its entropy drops by N × log N . Hence the entropy of the final sorted array is N × log 400 − N × log N . For N = 100, this expression evaluates to 200 bits. Because the final values of variables c and d are determined by the program, they are not counted in the entropy of the final state; we only count the entropy of the array and that of variables p and swap. We find that 200 + log(101) + log(400) = 213 bits. Hence the redundancy of the final state is 3328 − 213 bits = 3115 bits. Therefore, the state redun- dancy of this program can be represented by the interval: [2435 bits … 3115 bits]; the state redundancy of the program evolves from 2435 initially to 3115 in the final state. State redundancy measures the bandwidth of assertions that can be checked between variables of the state; in the final state it is almost 10 integers’ worth of assertions (i.e., nearly 10 assertions that equate 2 integer expressions to each other).

14.7.4 Functional Redundancy

According to the definition of functional redundancy, the functional redundancy of the sort program is obtained by the following formula,

HY Hρ − 1,

where Y is the output space of the program and ρ is the (random variable representing the) range of the program. For the output space, we let Y be the space of the S program, whose entropy we have found to be 3328 bits. For the range of the program, we have computed its entropy in the previous section and found it to be 213 bits. Hence the functional redundancy of this program is:

14.7 AN ILLUSTRATIVE EXAMPLE 329

Much of this redundancy stems from the fact that we are using 32-bit words (100 of them) to represent integers between 1 and 400 only (when 9 bits would have been sufficient). Another factor that further reduces the entropy of the output (hence increases its redundancy) is that the output array is sorted; as a result, each cell of the array limits the range of possible values for the remainder of the array (e.g., if the first cell is 200, then the remainder of the array is restricted to the range [201...400] rather than [1...400]).

14.7.5 Non-injectivity

The non-injectivity of a program is the conditional entropy of its input (or initial state) given its output (or final state). Because this program sorts arrays, we know that each sorted array of size N stems from N ! distinct initial arrays (assuming, as we do, that all array cells are distinct). If we take the hypothesis of uniform probability distribution, and if we focus on the array (rather than auxiliary index variables), we find that this conditional entropy is log(N !). According to Stirling’s approximation, this quantity can be approximated by

N × log N −N = 564 bits,

for N = 100.

14.7.6 Non-determinacy

While all the previous metrics reflect properties of the program, non-determinacy reflects properties of the specification that the program is intended to satisfy and against which it is judged and tested. We consider three possible specifications for the sort program and compute the non-determinacy of each:

• Ord, which specifies that the output is ordered. • Prm, which specifies that the final array is a permutation of the initial array. • Sort = Ord Prm.

According to the definition of non-determinacy, we must compute the conditional entropy H Y X , where X is the random variable that takes its values over the domain of the specification and Y is the random variable that ranges over the image set of X by R (i.e., such that X, Y R). We find the following:

• Ord: This specification provides that the output array is sorted but bears no relation to the input array. In this case, H Y X = H Y . The entropy of a sorted array of size N whose values may take V values (in our case V = 400) is given by the formula N × W − N × log N . For N = 100 and V = 400, we find

H Y = 2536 bits. This is the extent to which a candidate program can deviate from the function of sorting and still not violate the specification Ord.

330 METRICS FOR SOFTWARE TESTING

• Prm: This specification provides that the final array is a permutation of the initial array but it does not stipulate that it must be ordered. For an array of size N (whose cells we assume to be distinct), there exist N! distinct arrays that satisfy this specification. Under the assumption of uniform probability, this entropy is

H Y X = log N , which we estimate by means of the Sterling approximation as N × log N −N = 564 bits. This is the extent to which a candidate program can deviate from the function of sorting and still not violate the specification Prm.

• Sort: Because this specification is deterministic, H Y X = 0. Any deviation of a can- didate program from the mission of sorting the array will violate this specification.

14.7.7 SUMMARY

The following table summarizes the discussion of the sorting routine’s metrics.

Interpretation/meaning/relation to testing

Cyclomatic complexity 5 Fault proneness Volume

Fault proneness State Redundancy

Bandwidth of assertions that can check for

errors, in initial state and final state Functional Redundancy

14.62 Factor by which state space is larger than the result of the computation

Bandwidth of data that the program may lose and still recover spontaneously

Amount of data a program can lose and Determinacy

still manage to satisfy the specification Prm

Sort