TESTING MATURITY MODEL

18.4 TESTING MATURITY MODEL

Similar to the concept of evaluating and improving software development processes, there is a need for a framework to assess and improve testing processes. Continuous improvement of testing processes is an ideal goal of organizations, and evaluation plays a key role in process improvement. This is because an organization must know its present level of maturity before it takes actions to move to the next level. Ilene Burnstein and her colleagues at the Illinois Institute of Technology pioneered the concept of the TMM to help organizations evaluate and improve their testing processes [5, 6].

The TMM describes an evolutionary path of test process maturity in five levels , or stages, as illustrated in Figure 18.3. The TMM gives guidance concerning how to improve a test process. Each stage is characterized by the concepts of

Level 5: Optimization/defect prevention and quality control

Test process optimization Quality control Application of process data for defect prevention

Level 4: Management and measurement

Software quality evaluation Establish a test measurement program Establish an organizationwide review program

Level 3: Integration

Control and monitor the testing process Integrate testing into the software life cycle Establish a technical training program Establish a software test organization

Level 2: Phase definition

Institutionalize basic testing techniques and methods Initiate a test planning process

Develop testing and debugging goals

Level 1: Initial

Figure 18.3 Five-level structure of TMM. (From ref. 5. © 2003 Springer.)

569 maturity goals , supporting maturity goals, and activities, tasks, and responsibilities

18.4 TESTING MATURITY MODEL

(ATRs), as explained in the following: • Maturity Goals: Each maturity level, except level 1, contains certain

maturity goals. For an organization to achieve a certain level of matu- rity, the corresponding maturity goals must be met by the organization. The maturity goals are specified in terms of testing improvement goals.

• Maturity Subgoals: Maturity goals are supported by maturity subgoals. To achieve a maturity goal, it may be necessary to meet several, fine-grained subgoals.

• Activities, Tasks, and Responsibilities: Maturity subgoals are achieved by means of ATRs that address issues concerning implementation of activ- ities and tasks. ATRs also address how an organization can adapt its practices so that it can move in-line with the TMM model, that is, move from one level to the next. ATRs are further refined into “views,” known as critical views, from the perspectives of three different groups of people: managers, developers and test engineers, and customers (users/clients).

The maturity goals associated with the five levels of the TMM model will be explained in the following.

Level 1. Initial No maturity goals are specified at this level. The TMM level 1 is called the initial level. For an organization to be at level 1, nothing special needs to

be done. Level 1 represents a scenario where testing is not performed in a planned manner. Testing begins after code is written. At this level, an organization often performs testing to demonstrate that the system works. No serious effort is made to track the progress of testing and the quality level of a product. Test cases are designed and executed in an ad hoc manner . Testing resources, such as trained testers, testing tools, and test environments, are not available. In summary, testing is not viewed as a critical, distinct phase in software development.

Level 2. Phase Definition At level 2, the maturity goals are as follows: • Develop testing and debugging goals.

• Initiate a test planning process. • Institutionalize basic testing techniques and methods.

(i) Develop Testing and Debugging Goals Separation of testing from debugging is an important growth in the maturity of a testing process. Unit testing and debugging may have some common features such as those being performed by individual programmers. However, their separation becomes more evident when we consider higher levels of testing, for example, integration testing, system-level testing, and acceptance testing. As we move away from unit-level testing to system-level testing, for example, test engineers are more interested in examining the higher level features of the system, and they do not focus on code-level details. On the other hand, debugging is primarily a code-level activity