TESTING BY SYMBOLIC EXECUTION

12.4 TESTING BY SYMBOLIC EXECUTION

When we deploy a test driver on some test data and the oracle is satisfied, the only evidence of correct behavior that we have collected pertains to the precise test data on which the candidate program was tested; whether the test driver relies on randomly generated test data, or on targeted, pre-generated test data, the space covered by test

270 TEST DRIVER DESIGN

data is typically a very small fraction of the domain of the specification. To overcome this limitation, it is possible to simulate the execution of a program without commit- ting to a particular value of the input; to this effect, we represent the input values by symbolic names, rather than actual concrete values and analyze the effect of executing the program on these values, so as to compare them with the requirements imposed by the specification. For all intents and purposes, this is essentially a static verification method, but it is considered as part of the toolbox of the software tester; we refer to this technique as symbolic testing because it consists in effect in testing a program by executing it symbolically (rather than actually) on symbolic data (rather than actual concrete data). Whereas actual program execution produces an actual output for a spe- cific actual input value, symbolic execution produces a symbolic expression of the output as a function of a symbolic representation of the input; this amounts, in effect, to computing the function of the program. In Chapter 5, we had talked about program functions without discussing how these are derived; in this section, we briefly discuss how this can be done in a bottom-up stepwise process, which proceeds inductively on the program structure.

We can think of a program function as mapping inputs (from an input stream, say) onto outputs (stored in an output stream); but very often, it is more interesting and more convenient to think of a program function as mapping initial states to final states. To accommodate these two perspectives without too much complexity overhead, we generally focus on state transformation, but we may sometimes (especially when we discuss I/O operations) assume that we have a default input stream (is) and a default output stream (os) as part of the state space. We consider a simple C-like programming language, and we consider, in turn, its elementary statements and then its compound statements.

Elementary statements include assignment statements and input/output statements (which we denote respectively by read()and write()). We denote by S the space of the program (whose function we are computing), and by s and s arbitrary states of the program.

• Assignment Statement. Let x be a variable of some type T, let E be an expression on S that returns a value of type T, and let def(E) be the set of states on which expression E is defined (can be computed). Then

x=Es= s,s s def E x=Ex −s=−s, where − s respectively − s designates all the variable names in s (respec-

tively s ) other than x. • Read Statement. Let x be a variable of type T, and let is (the default input stream)

be structured as a sequence of T-type values. Then,

read x = s,s length is > 0 x = head is is = tail is −s=−s,

12.4 TESTING BY SYMBOLIC EXECUTION 271

where − s respectively − s designates all the variable names in s (respec- tively s ) other than x and is.

• Write Statement. Let x be a variable of type T, and let os be the default output stream of the program. Then,

write x = s,s os = os x −s=−s, where − s respectively − s designates all the variable names in s (respec-

tively s ) other than os and designates concatenation. Compound statements include the structured control constructs of imperative pro-

gramming languages, most notably: • The Sequence Statement, whose rule is defined as follows:

g 1 ;g 2 =g 1 •g 2 ,

where designates the relational product. • The Conditional Statement, whose rule is defined as follows:

if t g 1 =It•g 1 I ¬t ,

where I t = s,s s=sts. • The Alternative Statement, whose rule is defined as follows:

if t g 1 else g 2 =It•g 1 I ¬t • g 2

• The Iterative Statement, whose rule is defined as follows:

while t b=It•b • I ¬t

Because the formula of the while rule is difficult to apply in practice, we have a theorem that characterizes such functions.

Theorem: (due to H.D. Mills (1975)) Let w: while (t) {b} be a while statement on space S and let W be a function on S. Then w computes function W if and only if the following conditions are satisfied:

1. dom(W) is the set of states on which the loop terminates normally.

2. I ¬t • W = I ¬t .

3. I t • W = I t • B • W.

272 TEST DRIVER DESIGN

In order to apply this theorem, we need to derive function W based on our under- standing of what the loop does, then check that W verifies the conditions set forth above. We illustrate this theorem with two simple examples.

Let S be defined by variables x and y of type integer and let w be the following loop on S:

w: while (y!=0) {x=x+1; y=y-1;}.

We consider the following function W:

W= s,s y ≥ 0 x = x + y y = 0

The first condition of the theorem is satisfied, since the domain of W is the set of states for which y is nonnegative, and that is exactly the set of states for which the loop terminates. As for the next two conditions, we check them briefly below:

I ¬t • W = {substitution} s,s y = 0 s = s

s, s y≤0 x=x+yy=0

= {pre-restriction} s,s y = 0 y ≥ 0 x = x + y y = 0 = {simplification} s,s y = 0 x = x y = y = {substitution}

I ¬t As for the third condition, we write It•B•W = {substitution, pre-restriction}

s,s y 0 x=x+1 y = y −1 s,s y ≥ 0 x=x+y y=0 = {relational product} s,s y 0y−1≥0 x=x+1+y−1 y=0

12.4 TESTING BY SYMBOLIC EXECUTION 273

= {simplification} s,s y > 0 x = x + y y = 0 = {pre-restriction} Iy 0• s,s y ≥ 0 x=x+y y=0 = {substitution} It•W As a second example, we let S be defined by natural variables n, f, k, and we con-

sider the following loop on space S:

w: while(k!=n+1) {f=f*k; k=k+1;}.

We let W be the function on S efined by:

W= s,s k ≤ n + 1 n = n f = f × k=n+1

k −1

The domain of W is the set of states such that k ≤ n + 1, which is precisely the set of states on which the loop terminates. We check in turn the two remaining conditions of the theorem, as follows:

I ¬t • W = {substitution}

n s,s k = n + 1 s = s • s,s k ≤ n + 1 n = n f = f ×

k=n+1 k −1

= {pre-restriction}

s,s k = n + 1 n = n f = f ×

k=n+1

k −1

= {simplification} s,s y = 0 x=xy=y = {substitution}

I ¬t

274 TEST DRIVER DESIGN

As for the third condition, we write It•B•W = {substitution, pre-restriction}

s,s k n+1 n=nf=f×kk=k+1 •

s, s k ≤ n + 1 n = n f = f ×

k=n+1

k −1

= {relational product}

s, s k n+1k≤n+1n=n f=f×k× k=n+1 k + 1 −1

= {simplification}

s, s k < n + 1 n = n f = f ×

k=n+1

k−1

= {pre-restriction}

Ik n+1• s,s k ≤ n + 1 n = n f = f × k=n+1

k −1

= {substitution} It•W