TESTING TAXONOMY

7.3 TESTING TAXONOMY

In this section, we consider a number of different test activities, analyze them, and discuss to what extent the classification scheme presented in this chapter enables us to characterize them in a meaningful manner.

7.3.1 Unit-Level Testing

We distinguish between two types of unit-level testing: • Unit-Level Fault Removal: This test is carried out by the unit’s developer as part

of the coding and unit testing phase of the software life cycle; its purpose is to detect, isolate, and remove faults as part of the development life cycle.

• Unit-Level Certification: This test is carried out by the configuration manage- ment/quality assurance team for the purpose of ensuring that the unit under test meets the quality standards mandated for the project.

The following table illustrates how these two tests differ from each other, by com- paring and contrasting their attributes.

7.3 TESTING TAXONOMY 137

Attributes

Unit-level certification Scale

Unit-level fault removal

Unit (module, routine,

Unit (module, routine,

Finding and removing faults

Certifying compliance with project-wide quality

ributes

standards

att Property

Functionality

Functionality

rimary Method

Structural (attempting to

Functional (attempting

sensitize and expose as

to exercise as many

many faults as possible)

functional aspects as possible)

Oracle

The function that the unit is

The specification that

designed to compute

the unit is designed to satisfy

Test life cycle

Semisequential

Sequential (generate test data, run the unit on the test data, and record outcomes, rule on certification)

Test assumptions

The intended function is not

The unit specification

in question (the correctness

is not in question

of the unit is)

(the correctness of the unit is)

Completion criterion

Confidence that most

Confidence that the unit

ributes

egregious faults have been

has passed/ or has

att

removed

failed the certification standard

ndary Required artifacts

Executable code

Executable code

+ Source code

+ test environment

Seco

+ test environment

+ Unit specification

+ Intended function

Test stakeholders

Unit developer

Unit developer + Configuration management/quality Assurance team

Test environment

Simulated environment

Existing (evolving) system + Simulated environment

Position in the SW life

During the programming

Concludes the

cycle

phase

programming phase for each individual unit

A SOFTWARE TESTING TAXONOMY

7.3.2 System-Level Testing

We consider three system-level tests:

1. Integration test, which arises at the end of the programming phase, when programming units that have been developed, tested, and filed into the product configuration are combined according to the product design to produce an inte- grated product.

2. Reliability test, which arises as the end of the software development project, prior to product delivery, to evaluate the reliability of the product (and eventu- ally, ascertain that the product reliability exceeds the product’s required reliability).

3. Acceptance test, which is conducted jointly by the development team and the user team to check that the software product meets its requirements.

Even though these tests are all at the same scale (system-wide), they differ from each other in significant ways, as we see in the table below.

Attributes

Acceptance test Scale

Integration test

Reliability test

Find and remove

Assess the reliability Check whether

design faults

of the product

the system meets

(dealing with

its requirements to

inter-component

the satisfaction of

attributes

coordination)

the user

ary Property

Prim Method

Structural

Functional (as per (compatible with

Functional

user requirements)

usage pattern)

Oracle

System function

System

System Specification (or

specification subspecification with respect to which we want to estimate reliability)

Test life cycle

attributes ry

Test

Units are not in

Test environment Test environment

assumptions

question; only

mimics operating mimics operating

system design is

environment

environment

econda S Completion

All relevant

obligation met

exercised, all

estimated/ or

possible faults

reliability

removed

requirement met

Executable code +

Executable code + Executable code +

artifacts

source code +

usage pattern + contractual

expected function

Test

Product designers

Product designers + Requirements

stakeholders

verification and

engineers + user

validation

representative + managers

environment (or environment (or simulation thereof )

simulation thereof )

Position in the

Integration

Prior to delivery At delivery

SW life cycle