Hybrid Validation

6.4.2 Hybrid Validation

According to the foregoing discussion, it is advantageous to deploy more than one method of verification, by virtue of the law of diminishing returns; according to this law, each method offers high returns on some aspects of verification and lower returns on others. If we deploy different methods that offer high returns on complementary aspects, then we can afford the luxury of applying each method where its returns are high. As a simple illustration of this philosophy, we consider the following algorithm for selection sort. While this algorithm is fairly straightforward, one may be forgiven for having some doubt that it is correct; some examples are as follows: Are all the indices initialized properly? Are they incremented properly? Are they tested properly, against the right boundaries? Are the inequalities correct (strict or large)? Does this algorithm work for N=1?

void somesort (itemtype a[MaxSize], indextype N) // line 1 {

// 2 indextype i; i=0;

// 3 while (i<=N-2)

// 4 {indextype j; indextype mindx; itemtype minval;

// 5 j=i; mindx=j; minval=a[j];

// 6 while (j<=N-1)

// 7 {if (a[j]<minval) {mindx=j; minval=a[j];}

// 8 j++;}

// 9 itemtype temp;

// 10 temp=a[i]; a[i]=a[mindx]; a[mindx]=temp;

To lift these doubts, one may want to use the technique discussed in Chapter 5; to this effect, we would have to derive a precondition and a postcondition for this routine. To check that the function sorts array a, we would write the following precondition/ postcondition pair:

• Precondition: ϕ(s) ≡ (s=s0). • Postcondition: ψ(s) ≡ prm(s,s0)

sorted(s),

where prm(s,s0) means that array a is a permutation of array a0, and sorted(s) means that array a is sorted.

Applying the proof technique presented in Chapter 5 involves a few pages of proof, and requires the invention of several intermediate assertions, and two invariant asser- tions, one for each loop. Any wrong choice of an intermediate assertion or an invariant assertion will leave us in limbo, unsure whether the program is incorrect or our choice of assertion is inappropriate. On the other hand, if we wanted to test this program using the same specification as an oracle, then we would have to write a Boolean function

6.4 FAULT MANAGEMENT 119

that checks whether the final array is a permutation of the initial array; such a Boolean function is difficult to write and is perhaps more complex and more error prone than writing the sort algorithm itself.

As an alternative, we choose to prove the correctness of the program with respect to prm(s,s0) and to test the program using the oracle sorted(s). As far as proving the cor- rectness of the program with respect to prm(s,s0), it suffices to consider that the only location in the program where array a is changed at all, is line 11, where cells a[i] and a [mindx] are swapped. Swapping two cells of an array preserves the property prm (s,s0), hence this specification is satisfied. Specifically, we would define the following precondition/postcondition pair:

• Precondition: ϕ(s) ≡ (s=s0), • Postcondition: ψ(s) ≡ prm(s,s0),

which would yield the following verification condition: v: {s=s0}

indextype i; i=0; while (i<=N-2) {indextype j; indextype mindx; itemtype minval;

j=i; mindx=j; minval=a[j]; while (j<=N-1) {if (a[j]<minval) {mindx=j; minval=a[j];} j++;} itemtype temp; temp=a[i]; a[i]=a[mindx]; a[mindx]=temp; i++;}

{prm(s,s0)} By virtue of the consequence rule, we derive the following verification condition: v0: {prm(s,s0)}

indextype i; i=0; while (i<=N-2) {indextype j; indextype mindx; itemtype minval;

j=i; mindx=j; minval=a[j]; while (j<=N-1) {if (a[j]<minval) {mindx=j; minval=a[j];} j++;} itemtype temp; temp=a[i]; a[i]=a[mindx]; a[mindx]=temp; i++;}

{prm(s,s0)}

120 FAILURES, ERRORS, AND FAULTS

We can easily verify this formula, using prm(s,s0) as an intermediate assertion and an invariant assertion throughout the program (it is preserved vacuously since most statements do not affect array a), until we generate the verification condition: w: {prm(s,s0)} temp=a[i]; a[i]=a[mindx]; a[mindx]=temp; {prm(s,s0)} which we assume to hold since all it does is to permute two cells of array a.

Now, to test this routine for specification sorted(s), we write the following test driver:

#include <iostream> #include <fstream> #include <string> #include <iomanip> #include "rand.cpp" using namespace std;

// types typedef int itemtype; typedef int indextype;

// constants const indextype MaxSize = 60; const itemtype ValueRange = 200; const int TestLength = 10000;

// variables, current state indextype N; itemtype a [MaxSize];

// working variables int testnumber=0; // counting tests int nbfail = 0;

// counting failures

string tf="testfile.dat"; // function headers

int drawint (int ValueRange); void generatetestdata (itemtype a [MaxSize], indextype& N); bool moretests (); void testreport();

// program and specification/ oracle void somesort (itemtype a[MaxSize], int N); // any sorting

algorithm bool Oracle(itemtype a[MaxSize], indextype N);

6.5 CHAPTER SUMMARY 121

fstream teststream (tf.data(), ios::out); int main ()

{SetSeed(684); // random value Iwhile (moretests())

{generatetestdata (a,N); somesort(a,N); if (!Oracle(a,N)){nbfail=nbfail+1;}};

testreport();} int drawint (int ValueRange)

{int val; val = 1+ int(NextRand()*ValueRange); return val;}

void generatetestdata (itemtype a [MaxSize], indextype& N) {N = drawint(MaxSize); for (int k=0; k<N; k++) {a[k]=drawint(ValueRange);}}

bool Oracle(itemtype a[MaxSize], indextype N) {bool sorted; sorted=true; for (int k=0; k<N-1; k++) {sorted = sorted && (a[k]<= a[k+1]);}; return sorted;}

The oracle of this test is very simple, since it merely checks that the current array is sorted. Also because this test driver is written on the basis of the specification of a sorting algorithm, it can be applied to any sorting function; it is not specific to the selection sort.

Execution of the test driver on the selection sort function, generating ten thousand random arrays of random size between 1 and 60, of random values between 1 and 200, yields no failure. While we have not proven that the program has no faults, we have shown that if it does have faults, they do not appear to be causing frequent failures; this does not prove that the program never fails but makes it unlikely that failure is fre- quent. We have reached this conclusion with relatively little effort.