ACCEPTANCE TEST PLAN

14.4 ACCEPTANCE TEST PLAN

Planning for acceptance testing begins as soon as the acceptance criteria are known. Early development of an acceptance test plan (ATP) gives us a good picture of the final product. The purpose of an ATP is to develop a detailed outline of the process to test the system prior to making a transition to the actual business use of the system. Often, the ATP is delivered by the vendor as a contractual agreement, so that the business acceptance testing can be undertaken within the vendor’s development organization to ensure that the system eventually passes the acceptance test.

462 CHAPTER 14 ACCEPTANCE TESTING

In developing an ATP, emphasis is put on demonstrating that the system works according to the customer’s expectation, rather than passing a set of com- prehensive tests. In any case, the system is expected to have already passed a set of comprehensive tests during system-level testing. The ATP must be kept very simple because the audience of this plan may include people from diverse backgrounds, such as marketing and business managers. Some people argue that the ATP is redundant and unnecessary if a comprehensive system test plan is developed. We believe that even if a system test plan is adequate, acceptance tests usually uncover additional significant problems. Moreover, user’s concerns are not addressed during system-level testing.

An ATP needs to be written and executed by the customer’s special user group. The user group consists of people from different backgrounds, such as software quality assurance engineers, business associates, and customer support engineers. In addition, the acceptance test cases are executed at the user’s opera- tional environment, whereas the system-level test cases are executed in a laboratory environment. An overall test plan for acceptance testing and description of specific tests are documented in the ATP. The structure of a typical ATP is outlined in Table 14.1.

The introduction section of the ATP describes the structure of the test plan and what we intend to accomplish with this test plan. This section typically includes (i) test project name, (ii) revision history, (iii) terminology and defini- tions, (iv) names of the approvers and the date of approval, (v) an overview of the plan, and (vi) references.

For each quality category from the acceptance criteria signed-off document two subsections are created: operational environment and test case specification. The operational environment deals with discussion on site preparation for the exe- cution of the acceptance test cases. Test cases are specified for each acceptance criteria within the quality category.

An outline of the timeline of execution of acceptance tests is provided in the schedule section of the ATP. Acceptance test execution is not intended to

be exhaustive, and therefore it does not continue for long. The acceptance test

TABLE 14.1 Outline of ATP 1. Introduction

2. Acceptance test category. For each category of acceptance criteria: (a) Operational environment (b) Test case specification

(i) Test case ID number (ii) Test title (iii) Test objective (iv) Test procedure

3. Schedule 4. Human resources

463 may take up to six weeks for a large system. The point here is that comprehensive

14.5 ACCEPTANCE TEST EXECUTION

acceptance testing, to the same extent and depth as targeted by system-level testing, is not required to demonstrate that the acceptance criteria are satisfied by the system.

The human resources section of the ATP deals with (i) the identification of the acceptance testers that form the client organization and (ii) their specific roles in the execution of acceptance test cases. The section includes acceptance test site preparation, overseeing installation of new hardware, upgrading the software, and setting up of the networks. These are the people who are knowledgeable in the operational environment and business operations. In addition, the human resources requirement from the supplier organization during the acceptance testing is included in this section. These engineers are usually from the supplier’s system test group, who participated in testing the system.

The ATP is reviewed and approved by the relevant groups, such as the mar- keting, customer support, and software quality assurance groups. It can be shared with the system supplier organization.