Review of deliverables (documents) and acceptance tests

12.5.9 Review of deliverables (documents) and acceptance tests

Two of the most powerful tools for assuring the quality of external partici- pants, primarily subcontractors and customer-supplied software, are thorough review of software development documents (“deliverables”) by the contractor and acceptance tests, planned, designed and carried out by the contractor. These tools provide independent and direct review of develop- ment documents and testing of the software components of the external participant’s products.

It has been suggested that the presence of the subcontractor’s represen- tative in design review teams could replace the contractor’s independent review of deliverables and acceptance tests. In many cases, cost and timetable considerations force contractors to be satisfied with participation in the sys- tem testing process carried out by the subcontractor. Decisions regarding these options should be taken very carefully.

Summary

(1) Explain the difference between contractors and external participants.

Software development contractors are organizations or groups of organizations that are contracted by a customer in a project contract to develop a software system. External participants are organizations that participate in the development process, performing small to large parts of the work, but are not legally designated sides in the project’s contract.

(2) List the types of external participants, and explain the benefits they provide to the contractor.

The external participants can be categorized into three main groups:

As

■ Subcontractors

suring the quality

■ Suppliers of COTS software and reused software modules ■

The customer themselves as an active participant in performing parts of the project.

The main benefits to the contractor of using external organizations are:

■ Overcoming shortages of professional staff by transferring parts of the project

to be carried out to firms employing staff having those skills.

■ Potentially shorter project schedules, achieved by purchasing COTS software

of

and reused software rather than developing the software.

extern

■ Expertise acquired in areas that need specialization through the participation of owners – the subcontractor or the customer’s development department – of that expertise.

al

■ Saved budget, achieved when subcontractors offer prices below those incurred by

a performing the project internally, and by the use of COTS and reused software. rtic

ipants’ c

(3) Describe the risks for the contractor associated with turning to external participants.

■ Delays in completion of the project parts due to the external participants’ com- peting interests, given that the contractor is the only party committed to living

ontribution

up to schedule demands as stated in the project contract signed with the cus- tomer. Even the customer – as supplier for their own project – might prefer another project and delay completion of their part.

■ Low quality of project parts caused by insufficient capabilities, attempts to save resources, or other factors.

■ Future maintenance difficulties due to low quality or non-standard software and/or incomplete or poor documentation of parts carried out by external participants.

■ Loss of control over parts of the project instigated by periods of cut-off com-

munication, whether intentionally or inadvertently initiated.

(4) List the SQA tools appropriate for use with external participants and add short statements regarding the risks they help to eliminate or reduce.

■ The requirements document review assures a correct and complete list of the requirements related to software functionality, to formal and staff aspects of the project and to SQA issues; it contributes mainly to delay reduction and fewer cases of low quality.

■ Choice of external participants, if done properly, reduces all types of risks, especially those related to low quality.

■ The project coordination and joint control committee, if it operates properly, discovers anticipated as well as unanticipated delays, quality problems and potential for loss of control over the project at early stage. Early alerts and coop- eration can reduce and even eliminate these risks. The committee’s suggested solutions to quality problems, especially those related to documentation, also

■ Participation in design reviews provides an excellent opportunity to examine

the real quality of a subcontractor’s work and to introduce corrections where necessary. The review’s main contribution is to low-quality product reduction.

view ques

■ Participation in software testing, like participation in design reviews, con-

tributes to the reduction of low-quality products. Furthermore, such participation enables the contractor to be alerted about possible delays in the work schedule, and can help reduce the effects of those delays.

■ Specialized procedures are part of the contractors’ SQA infrastructure and are

tion

meant to handle all kinds of risky situations. ■

Qualification and certification of the supplier’s team leaders and other staff

assure the professional capacities of the project teams, and contribute to reduc- tion of low-quality products.

■ Progress reports of external participants’ development activities are prepared mainly in order to reduce the risk of delays.

■ Review of deliverables (documents) and acceptance tests is aimed at assuring the quality of the work performed by the external participant and consequently reduces the risks of future maintenance difficulties.

Selected bibliography

1. Basili, V. R. and Boehm, B. (2001) “COTS-based systems Top 10 list”, Computer, 34(5), 91–93.

2. IEEE (1998) “IEEE Std 1062-1998–IEEE Recommended Practices for Software Aqcuisition”, in IEEE Software Engineering Standards Collection, The Institute of Electrical and Electronics Engineers, New York. 3. ISO (1997) ISO 9000-3:1997(E), Quality Management and Quality Assurance Standards – Part 3: Guidelines for the Application of ISO 9001:1994 to the Development, Supply, Installation and Maintenance of Computer Software, 2nd edn, International Organization for Standardization (ISO), Geneva. 4. ISO/IEC (2001) “ISO 9000-3:2001 Software and System Engineering – Guidelines for the Application of ISO 9001:2000 to Software, Final draft”, International Organization for Standardization (ISO), Geneva, unpublished draft, December 2001. 5. Oskarsson, O. and Glass, R. L. (1996) An ISO 9000 Approach to Building Quality Software, Prentice Hall, Upper Saddle River, NJ.