Specification Validation

4.3.4 Specification Validation

The software engineering literature is replete with examples of software projects that fail, not because programmers do not know how to write code or how to test it, but rather because analysts and engineers fail to write valid specifications, that is, speci- fications that capture all the relevant requirements (for the sake of completeness) and nothing but relevant requirements (for the sake of minimality). Consequently, it is important to validate specifications for completeness and minimality and to invest the necessary resources to this effect before proceeding with subsequent phases of the software lifecycle. In this section, we briefly and cursorily discuss the process of specification validation, in the narrow context of the relational specifications that we introduce in this chapter, with the modest goal of giving the reader some sense of what it may mean to validate a specification.

Let us start with a very simple illustrative example: We consider space S defined by natural variables x and y, and we consider the following requirement:

Increase x while preserving the sum of x and y.

54 SOFTWARE SPECIFICATIONS

We submit the following relations as possible specifications for this requirement:

1. N 1 = s,s x≤x x+y=x+y

2. N 2 = s,s x+y=x+y y≥y

7. N 7 = s,s x=x+1x+y=x+y

8. N 8 = s,s x+y=x+y y = y−2 We invite the reader to ponder the following questions: which of these specifica-

tions is complete; and for those that are complete, which are minimal. The following table shows our answers to these questions (if a specification is not complete, it makes no sense to check its minimality):

Specification

Minimal? Valid? N 1 = s ,s x≤x x +y=x+y

Complete?

N/A No N 2 = s ,s x +y=x+y y≥y

N/A No N 5 = s ,s x−x =y−y y >y

Yes Yes N 7 = s ,s x =x+1x+y=x+y

Yes

No No N 8 = s ,s x +y=x+y y = y −2

Specifications N 5 and N 6 are complete and minimal (and are identical, in fact); they specify that x must be increased while preserving the sum of x and y. Specifications N 1 and N 2 are not complete because they do not stipulate that x must increase (they allow it to stay constant); and specification N 4 is not complete because it fails to specify that the sum of x and y must be preserved. Specifications N 3 ,N 7 , and N 8 are complete but not minimal because they specify by how much x must be increased, which is not sti- pulated in the requirement.

In the example aforementioned, we wrote the specifications on the basis of the pro- posed requirement (to Increase x while preserving the sum of x and y) and we judged the completeness and minimality of candidate specifications by considering the same source, that is, the proposed requirement. If the same person or group is tasked with generating the candidate specifications and judging their validity (completeness and minimality), then the same biases that cause the person to write invalid specifications may cause him/ her to overlook the invalidity of their specification. The only way to ensure a measure of confidence in the validation of the specification is to separate the team that generates the specification from the team that validates it. To this effect, we propose the following two-team, two-phase approach:

4.3 SIMPLE INPUT OUTPUT PROGRAMS

Activity Phase

Specification Generation Specification Validation Specification Generating the specification from

Generating validation data from the

Generation

sources of requirements same sources of requirements Specification Updating the specification according to Testing the specification against the

Validation

feedback from the validation team validation data generated earlier

• The Specification Generation Phase: In the specification generation phase, the specification team generates the specification by referring to all the sources of requirements (requirements documents). Using the exact same sources, the valida- tion team generates validation data that it intends to test the specification against. We distinguish between two types of validation data, which are as follows: ○ Completeness properties: These are properties that the specification must have

but the validation team suspects the specification team may fail to record. ○ Minimality properties: These are properties that the specification must not have but the validation team suspects the specification team may record inadvertently. For the sake of redundancy, the specification team and the validation team must work independently of each other.

• The Specification Validation Phase: In the specification validation phase, the validation team tests the specification against completeness and minimality data generated in the previous phase, while the specification team updates the spec- ification if it turns out that it was not complete or not minimal.

It remains to discuss the following: what form does the validation data take, and how does one test a specification against the generated validation data. The answers to these questions are given in the following definitions.

Definition: Completeness Given a requirements document, a completeness property V is a relation that represents requirements information that candidate specifications must capture. A specification R is said to be complete with respect to

V if and only if R refines V. Implicit in this definition is that a good completeness property is one that has the

potential to detect an incomplete specification; in other words, a good completeness property is one that the validation team believes the specifier team may have overlooked.

Definition: Minimality Given a requirements document, a Minimality Property W is a relation that represents requirements information that candidate specifica- tions must not capture. A specification R is said to be minimal with respect to W if and only if R does not refine W.

Implicit in this definition is that a good minimality property is one that has the potential to detect a nonminimal specification; in other words, a good minimality

56 SOFTWARE SPECIFICATIONS

property is one that the validation team believes the specifier team may have inadvert- ently recorded in the specification.

Completeness and minimality are not absolute attributes but rather relative with respect to selected completeness and minimality properties, as provided in the follow- ing definition.

Definition: Validity

A specification R is said to be valid with respect to complete- ness properties V = V 1 ,V 2 , …, V n , and minimality properties W = W 1 ,W 2 , …, W m if and only if R is complete with respect to every element of V and minimal with respect to every element of W.

We admit without proof that if R refines all of V 1 ,V 2 , …, V n , then it refines their join. Hence, the range of valid specifications with respect to completeness properties V=V 1 ,V 2 , …, V n and minimality properties W = W 1 ,W 2 , …, W m is represented in the Figure 4.13. As a illustrative example of specification validation, consider the following requirement pertaining to space S defined by an array a[1..N] of some type, a variable x of the same type, and an index variable k, which we use to address array a: Given that x is known to be in a, place in k the smallest index where x occurs. This is a variation of the example discussed in Section 4.2.1.

4.3.4.1 Specification Generation Phase Examples of completeness proper- ties include the following:

1. If each cell of array a contains the index of that cell and if x=1, then k should

be 1.

V 1 = s,s

h1≤h≤Nah=h

x=1 k=1

Range of valid specifications

V 1 ⊕ V 2 ⊕ ... ⊕ V n

Figure 4.13 Range of valid specifications.

4.3 SIMPLE INPUT OUTPUT PROGRAMS

2. If array a contains 1 everywhere and x=1, then k should be 1.

V 2 = s,s h1≤h≤Nah=1 x=1k=1

3. If array a contains the sequence 1..N in increasing order and x=N, then k should

be N.

V 3 = s,s h1≤h≤Nah=h x=N k=N Examples of minimality properties include the following:

1. There is no requirement to preserve x. W 1 = s, s

h1≤h≤Nah=x x=x

2. There is no requirement to preserve a. W 2 = s,s

h1≤h≤Nah=x a=a

4.3.4.2 Specification Validation Phase So far, we have looked at the require- ments documentation, but we have not looked at candidate specifications; generating validation data independently of specification generation is important, for the sake of redundancy. Now, let us consider a candidate specification and check whether it is complete with respect to the completeness properties and minimal with

respect to the minimality properties. We consider specification F 2 , introduced in Section 4.2.1 as

F 2 = s,s h1≤h≤Nah=x ak=x h1≤h<kah x To prove that F 2 refines V 1 , we must prove that F 2 has a larger domain than V 1

and that the restriction of F 2 to the domain of V 1 is a subset of V 1 . We find the following:

F 2 L= s,s

h: 1 ≤ h ≤ N a h = x

V 1 L= s,s h1≤h≤Nah=h x=1 Clearly, V 1 L is a subset of F 2 L. We compute the restriction of F 2 to V 1 L, and we

find the following:

58 SOFTWARE SPECIFICATIONS

= {substitution} s,s

h1≤h≤Nah=h

x=1

h1≤h≤Nah=x ak=x h1≤h<kah x = {simplification} s, s

h1≤h<kah x = {logic simplification}

h1≤h≤Nah=h

x=1 ak=x

h1≤h<kah x = {logic simplification} s,s

s, s h1≤h≤Nah=h

x=1 k=1

h1≤h≤Nah=h

x=1 k=1

= {substitution}

We now consider the completeness property V 2 . To prove that F 2 refines V 2 , we must prove that F 2 has a larger domain than V 2 and that the restriction of F 2 to the

domain of V 2 is a subset of V 2 . We find the following:

F 2 L= s,s h1≤h≤Nah=x

V 2 L= s,s

h1≤h≤Nah=1

x=1 Clearly, V 2 L is a subset of F 2 L. We compute the restriction of F 2 to V 2 L, and we

find the following:

F 2 V 2 L = {substitution}

s,s h1≤h≤Nah=1

x=1

h1≤h<kah x = {simplification, redundancy}

h1≤h≤Nah=x

ak=x

s,s h1≤h≤Nah=1

1≤k≤N x=1

h1≤h<kah 1

4.3 SIMPLE INPUT OUTPUT PROGRAMS

= {logic} s,s

h1≤h≤Nah=1

x=1 k=1

= {substitution}

We now consider the completeness property V 3 . To prove that F 2 refines V 3 , we must prove that F 2 has a larger domain than V 3 and that the restriction of F 2 to the

domain of V 3 is a subset of V 3 . We find the following:

F 2 L= s, s h1≤h≤Nah=x

V 3 L= s,s

h1≤h≤Nah=h

x=N Clearly, V 3 L is a subset of F 2 L. We compute the restriction of F 2 to V 3 L, and we

find the following:

F 2 V 3 L = {substitution} s,s

h1≤h≤Nah=h

x=N

h1≤h<kah x = {simplification} s,s

h1≤h≤Nah=x

ak=x

h1≤h<kah x = {logic} s,s

h1≤h≤Nah=h

x=N ak=N

h1≤h<kah x = {simplification, redundancy} s,s

h1≤h≤Nah=h

x=N k=N

h1≤h≤Nah=h

x=N k=N

= {substitution}

We turn our attention to checking the minimality of F 2 with respect to W 1 and W 2 .

60 SOFTWARE SPECIFICATIONS

F 2 = s,s h1≤h≤Nah=x

h1≤h<kah x W 1 = s,s

ak=x

h1≤h≤Nah=x

x=x Because F 2 and W 1 have the same domain, the only way to prove that F 2 does not

refine W 1 is to prove that F 2 W 1 L is not a subset of W 1 . To this effect, we compute the following:

F 2 W 1 L ={F 2 and W 1 have the same domain}

which is not a subset of W 1 . Hence F 2 is minimal with respect to W 1 . We can prove, likewise, that it is minimal with respect to W 2 . Indeed, F 2 does not preserve x, nor a.