Advantages and disadvantages of black box testing

9.5.5 Advantages and disadvantages of black box testing

The main advantages of black box testing are:

Black box testing allows us to carry out the majority of testing classes, most of which can be implemented solely by black box tests. Of the test classes unique to black box testing, of special importance are system per- formance tests such as load tests and availability tests.

For testing classes that can be carried out by both white and black box tests, black box testing requires fewer resources than those required for white box testing of the same software package.

The main disadvantages of black box testing are:

Possibility that coincidental aggregation of several errors will produce the correct response for a test case, and prevent error detection. In other words, black box tests do not readily identify cases of errors that coun- teract each other to accidentally produce the correct output.

Absence of control of line coverage. In cases where black box testers wish to improve line coverage, there is no easy way to specify the parameters of the test cases required to improve coverage. Consequently, black box tests may not execute a substantial proportion of the code lines, which are not covered by a set of test cases.

Impossibility of testing the quality of coding and its strict adherence to the coding standards.

Summary 209 Summ

(1) Explain testing objectives.

One should distinguish between direct and indirect testing objectives. The direct

ar

objectives are:

■ To identify and reveal as many errors as possible in the tested software ■

To bring the tested software to an acceptable quality level ■

To perform the required testing in an efficient and effective way, within budget and scheduling limitations.

The indirect objective: ■

To supply records of software errors to be used for error prevention.

(2) Discuss the differences between the various testing strategies, their advantages and disadvantages.

There are basically two testing strategies: ■

“Big bang testing”: tests the software as a whole, once the completed package is available.

“Incremental testing”: tests the software piecemeal – software modules are test-

ed as they are completed (unit tests), followed by groups of modules composed of tested modules integrated with newly completed modules (integration tests). Once the entire package is completed, it is tested as a whole (system test).

There are two basic incremental testing strategies: bottom-up and top-down. In top- down testing, the first module tested is the main module, the highest level module in the software structure; the last modules to be tested are the lowest level mod- ules. In bottom-up testing, the order of testing is reversed: the lowest level modules are tested first, with the main module tested last. ■

Big bang vs. incremental testing. Unless the program is very small and simple, applying the “big bang” testing strategy presents severe disadvantages. Identification of error in the entire software package when perceived as one “unit” is very difficult and, in spite of the vast resources invested, is not very effective. Moreover, performing perfect correction of an error in this context is frequently laborious. Obviously, estimates of the required testing resources and testing schedule tend to be rather fuzzy. In contrast, the advantages of incre- mental testing, because it is performed on relatively small software units, yields higher percentages of identified errors and facilitates their correction. As a result, it is generally accepted that incremental testing should be preferred.

Bottom-up vs. top-down. The main advantage of the bottom-up strategy is its relative ease of performance, while its main disadvantage is the lateness of the stage at which it is possible to observe the program as a whole. The main advan- tage of the top-down strategy is the early stage at which it is possible to demonstrate the program as a whole, a condition that supports early identifica- tion of analysis and design errors. The main disadvantage of the approach is the comparative difficulty of its performance.

(3) Describe the concepts of black box testing and white box testing, and discuss their advantages and disadvantages.