MODELING A TEST DESIGN PROCESS

11.6 MODELING A TEST DESIGN PROCESS

Test objectives are identified from a requirement specification, and one test case is created for each test objective. Each test case is designed as a combination of modular components called test steps. Test cases are clearly specified so that testers can quickly understand, borrow, and reuse the test cases.

Figure 11.6 illustrates the life-cycle model of a test case in the form of a state transition diagram. The state transition model shows the different phases, or states, in the life cycle of a test case from its inception to its completion through the following states: create, draft, review, deleted, released, update, and deprecated. Certain actions are taken by the “owner” of the state, and the test case moves to a next state after the actions are completed. In the following, the states are explained one by one. One can easily implement a database of test cases using the test case schema shown in Table 11.6. We refer to such a database of test cases as a test factory.

A test case is put in this initial state by its creator, called the owner or creator, who initiates the design of the test case. The creator initializes the following mandatory fields associated with the test case: requirement_ids, tc_id, tc_title, originator_group, creator, and test_category. The test case is expected to verify the requirements referred to in the requirement_ids field. The originator_- group is the group who found a need for the test. The creator may assign the test case to a specific test engineer, including himself, by filling out the eng_assigned field, and move the test case from the create to the draft state.

Create State

Draft State The owner of this state is the test group, that is, the system test team. In this state, the assigned test engineer enters the following information: tc_author, objective, setup, test_steps, cleanup, pf_criteria, candidate_for_automa- tion, automation_priority. After completion of all the mandatory fields, the test engineer may reassign the test case to the creator to go through the test case. The

346 CHAPTER 11 SYSTEM TEST DESIGN

TABLE 11.6 Test Case Schema Summary Field

Description

tc_id Test case identifier assigned by test author (80 characters) tc_title

Title of test case (120 characters) creator

Name of person who created test case status

Current state of the record: create, draft, review, released, update, deleted, and deprecated

owner

Current owner of test case

eng_assigned Test engineer assigned to write test procedure objective

Objective of test case (multiline string) tc_author

Name of test case author (user name) originator_group

Group that originates test (performance testing group, functional testing group, scaling testing group, etc.)

test_category Test category name (performance, stress, interoperability,

functionality, etc.)

setup List of steps to perform prior to test test_steps

List of test steps

cleanup

List of posttest activities

pf_criteria

List of pass/fail criteria

requirement_ids List of references to requirements ID from requirement

database

candidate_for_automation If test can be/should be automated automation_priority

Automation priority.

review_action Action items from review meeting minutes approver_names

List of approver names

test case stays in this state until it is walked through by the creator. After that, the creator may move the state from the draft state to the review state by entering all the approvers’ names in the approver_names field.

Review and Deleted States The owner of the review state is the creator of the test case. The owner invites test engineers and developers to review and validate the test case. They ensure that the test case is executable, and the pass–fail criteria are clearly specified. Action items are created for the test case if any field needs a modification. Action items from a review meeting are entered in the review_actions field, and the action items are executed by the owner to effect changes to the test case. The test case moves to the released state after all the reviewers approve the changes. If the reviewers decide that this is not a valid test case or it is not executable, then the test case is moved to the deleted state. A review action item must tell to delete this test case for a test case to be deleted.

A test case in the released state is ready for execution, and it becomes a part of a test suite. On the other hand, a test case in the update state implies that it is in the process of being modified to

Released and Update States

347 enhance its reusability, being fine tuned with respect to its pass–fail criteria, and/or

11.7 MODELING TEST RESULTS

having the detailed test procedure fixed. For example, a reusable test case should be parameterized rather than hard coded with data values. Moreover, a test case should

be updated to adapt it to changes in system functionality or the environment. One can improve the repeatability of the test case so that others can quickly understand, borrow, and reuse it by moving a test case in the released–update loop a small number of times. Also, this provides the foundation and justification for the test case to be automated. A test case should be platform independent. If an update involves a small change, the test engineer may move the test case back to the released state after the fix. Otherwise, the test case is subject to a further review, which is achieved by moving it to the review state. A test case may be revised once every time it is executed.

Deprecated State An obsolete test case may be moved to a deprecated state. Ideally, if it has not been executed for a year, then the test case should

be reviewed for its continued existence. A test case may become obsolete over time because of the following reasons. First, the functionality of the system being tested has much changed, and due to a lack of test case maintenance, a test case becomes obsolete. Second, as an old test case is updated, some of the requirements of the original test case may no longer be fulfilled. Third, reusability of test cases tend to degrade over time as the situation changes. This is especially true of test cases which are not designed with adequate attention to possible reuse. Finally, test cases may be carried forward carelessly long after their original justifications have disappeared. Nobody may know the original justification for a particular test case, so it continues to be used.