Paulk, M. C. (1999) “Analyzing the conceptual relationship between ISO/IEC gement

a 22. Paulk, M. C. (1999) “Analyzing the conceptual relationship between ISO/IEC gement

15504 (Software Process Assessment) and the Capability Maturity Model for software, Proceedings of the 1999 International Conference on Software Quality, Cambridge, MA, 1–11. 23. Paulk, M. C. (2001) “Extreme programming from a CMM perspective”, IEEE

andard st

Software, 18(6), 19–25. 24. Paulk, M. C., Curtis B., Chrissis, M. B. and Weber, C. V. (1993) Capability Maturity Model for Software, Version 1.1, CMU/SEI-93-TR-24, ESC-TR-93-

s 177, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA.

25. Paulk, M. C., Weber, C. V. Curtis, B., Chrissis, M. B. (1995) The Capability Maturity Model: Guidelines for Improving the Software Process, Addison- Wesley, Reading, MA. 26. Pitterman, B. (2000) “Telcordia Technologies: the journey to high maturity”, IEEE Software, 17(4), 89–96. 27. Royce, W. (2002) CMM vs. CMMI: From Conventional to Modern Software Management, Rational Edge, Rational Software Inc., http//www.thera- tionaledge.com. 28. Schulmeyer, G. G. (1999) “Standardization of software quality assurance – where is it all going?”, in G. G. Schulmeyer and J. I. McManus (eds), Handbook of Software Quality Assurance, 3rd edn, Prentice Hall, Upper Saddle River, NJ, pp. 91–113. 29. TickIT (2001) The TickIT Guide, A Guide to Software Quality Systems Construction and Certification Using ISO 9001:2000, 5.0 edition, BSI DISC TickIT Office, London. 30. Tingey, M. O. (1997) Comparing ISO 9000, Malcolm Baldridge, and the SEI CMM for Software: A Reference and Selection Guide, Prentice Hall, Upper Saddle River, NJ. 31. Wigle, G. B. and Yamamura G. (1999) “SEI CMM Level 5: Boeing Space Transportation systems software”, in G. G. Schulmeyer and McManus J. I. (eds) Handbook of Software Quality Assurance, 3rd edn, Prentice Hall, Upper Saddle River, NJ, pp. 351–380.

Review questions

23.1 The introduction to Part VI presents the four main benefits of SQA/software engi- neering standards.

23.2 National and international SQA standards contribute to organizations due to their

distribution along three different ways. Describe these routes in your own words and explain their importance.

Topic

23.3 The introduction to Part VI presents classes of SQA standards.

for di

(1) Define the various classes of SQA standards.

(2) Explain the differences between the classes. 23.4 Section 23.1 presents classes of software quality management standards.

sc u

ss ion

(1) Explain the differences between the two classes.

(2) Compare the scope of the two classes and discuss their differences with respect to the goals of software quality assurance.

23.5 The evolution and diversification of the CMM methodology have produced sever- al specialized CMM products that were offered to the software industry. At a certain point, SEI moved toward creation of integrated CMM models.

(1) Explain the reasons for this move. (2) List some arguments against integration.

23.6 One of the main activities of the Bootstrap Institute is training and accreditation of assessors.

(1) Describe the accreditation process in your own words and explain the impor- tance of each level of the accreditation process. (2) Discuss the special role of assessors in implementing the Bootstrap methodology.

23.7 The SPICE project performed a comprehensive trial for the early versions of the ISO/IEC 15504 Standard. Explain, in your own words, the contribution of the trial to development of the standard.

Topics for discussion

23.1 Two ISO/IEC standards were completed during the last few years: a new version of 9000-3 and the TR version of 15504. Study these standards.

(1) Compare the 9000-3 requirements with the processes to be assessed accord- ing to 15504. Discuss differences in subject matter as well as approach. (2) Present three examples of standard subject to demonstrate your conclusions.

23.2 ISO/IEC 9000-3 serves as a certification standard for interested software devel- opment organizations throughout the world.

(1) The ISO and the IEC are neither capable of nor interested in carrying out cer-

(2) Describe, in your own words, certification of an organization. (3) Explain the unique importance of each stage of a certification audit.

23.3 CMM and CMMI are both composed of almost identical capability maturity mod-

Quality

els. While CMM bases its assessments on 18 key process areas, CMMI employs 24 process areas.

m (1) Explain the differences between the CMM and CMMI process areas in rela-

a tion to the respective subject matter. n

a (2) Indicate which of the capability levels have been substantially changed. gement (3) Can you characterize the observed changes?

23.4 Section 23.4.4 describes the CMM implementation experience of four organizations.

andard st

(1) Discuss the common experience of these organizations.

(2) What additional information (excluded from the section and the referenced papers) could be helpful for the successful completion of an evaluation?

23.5 Appendix 23B lists the ISO/IEC 15504 process areas. A parallel list of CMMI process

areas classified into capability maturity levels is provided in Appendix 23A. (1) Discuss the differences between the ISO/IEC 15504 and the CMMI process

areas in relation to their subject matter. (2) Regarding the CMMI capability levels, try to relate the ISO/IEC 15504 processes to the appropriate CMMI capability level. Indicate which of the CMMI capability levels reveal substantial differences in the allocated processes and which reveal similarity.

(3) Can you pinpoint differences between the models?

Appendix 23A CMMI process areas

Table 23A.1: CMMI process areas PA code

Process area name Process area description

Capability maturity level 1: Initial

— No process area is required.

Capability maturity level 2: Managed

RM Requirements Management Analyzes requirements of the project’s products and verifies that project plans and products of planned activities conform to requirements.

PP Project Planning Plans project activities: resources, schedules and outputs are to achieve approval and

Table 23A.1: Continued

PA code Process area name

Process area description

Appendi

Capability maturity level 2: Managed (Cont.)

SAM

Supplier Agreement Management Manages acquisition of products and services

from suppliers and subcontractors: contracts,

progress control and quality assurance.

23A

MA Measurement and Analysis Develops, initiates and completes measurements and analyses required to

CMMI proc

support management progress control and fulfill other information needs.

PPQA Process and Product Develops, implements and follows up Quality Assurance

application of quality assurance tools for processes and software products.

es

CM Configuration Management Develops implements and operates a

a re

configuration management system: assures

integrity of work products, configuration

status accounting, etc.

Capability maturity level 3: Defined

RD Requirements Development Prepares and analyzes product component requirements as required for development and cooperation with customers.

TS Technical Solution Introduces solutions to requirements: analyses, designs and implements the solutions for the product as a whole or its individual components.

PI Product Integration Integrates systems and product components developed by different teams into a completed product that functions according to the specified requirements.

VER Verification Assures that the product and its components comply with specifications.

VAL Validation Assures that the product and its components fulfill the customer’s actual use needs.

OPF Organizational Process Focus Develops and maintains the organization’s understanding of its process and procedures infrastructure and the activities for initiation of corrective actions and process improvements throughout the organization.

OPD Organizational Process Definition Develops and maintains an adequate infrastructure of organizational processes and procedures.

OT Organizational Training Identifies the skills and knowledge needs of people so they can effectively and efficiently

Table 23A.1: Continued

23 PA code Process area name

Process area description

Quality

Capability maturity level 3: Defined (Cont.)

IT Integrated Teaming Organizes stakeholder teams that collaborate to support the project team in achieving

m project goals.

n Implements continuous activities to identify

a RM

Risk Management

a gement project risks and potential risks, prevent

potential risks and eliminate or reduce damages.

DAR Decision Analysis and Resolution Establishes structured decision-making for selection of project implementation

andard st

alternatives, based on evaluation of alternatives according to defined criteria.

OEI Organizational Environment Establishes approach and infrastructure for for Integration

s integration of teams, implements said

approach.

Capability maturity level 4: Quantitatively managed

OPP Organizational Process Develops, implements and maintains Performance

quantitative projects quality and performance objectives that conform with organization’s objectives.

QPM Quantitative Project Management Applies quantitative management of project’s defined process and product metrics to control performance and identify improvement needs and opportunities.

Capability maturity level 5: Optimizing

OID Organizational Innovation Initiates development and implementation of and Deployment

selected incremental productivity and quality improvements in organization’s processes and technologies.

CAR Causal Analysis and Resolution Systematically operates corrective and preventive actions: analyzes failure and success data, develops and implements preventive and corrective actions, follows up results.

Source: Based on Compusol News, August 21, 2002, compusolsoftware.com/gocmmi.htm.

Appendix 23B ISO/IEC 15504 model processes

505 Appendi

Table 23B.1: ISO/IEC 15504 model processes Process

Process name

Process description

code

x 23B

CUS.1 Acquire software Activities required for a customer to obtain contracted software, including requirements specification, contracting the supplier,

ISO/IE

follow-up of development process (if any) and acceptance testing.

CUS.2 Manage customer needs Establish and maintain the customer’s

software requirements file; update files

15504 model

according to changing customer’s needs. CUS.3

Supply software Deliver and install at customer’s site a software package that conforms with all specified quality requirements.

CUS.4 Operate software Operate installed software correctly and efficiently.

proc

CUS.5 Provide customer service Provide an acceptable level of support services to enable effective use of software by the customers.

es ses

ENG.1 Develop system requirements Analyze and define system requirements; and design

allocate each requirement to a system element, including delay of implementation to later releases.

ENG.2 Develop software requirements Analyze and define system’s software requirements.

ENG.3 Develop software design Prepare a software design that fully complies with software requirements.

ENG.4 Implement software design Produce the software unit code, perform unit tests and complete necessary corrections.

ENG.5 Integrate and test software Perform unit integration and integration tests. Perform entire software integration and test the software system. Complete integration and software system corrections.

ENG.6 Integrate and test system Perform product system integration of software and non-software components. Perform system test and corrections of detected defects.

ENG.7 Maintain system and software Perform corrective maintenance according to user calls; perform adaptive and functional maintenance according to customer requests.

SUP.1 Develop documentation Develop and implement documentation procedure for process activities.

Table 23 B.1: Continued

23 Process Process name

Process description code

Quality

SUP.3 Perform quality assurance Establish software quality assurance system to assure that software products comply with requirements and standards.

a Perform work product verification Verify that each product of a process fully n

SUP.4

complies with its specified requirements.

a SUP.5 Perform work product validation gement Confirm that the work product fulfills the

requirements of the system’s intended user. SUP.6

Perform joint reviews Together with customer, maintain joint follow-up of contract implementation; reach

st understanding about required actions and andard

process changes to satisfy customer. SUP.7

Perform audits Perform independent reviews and audits of processes and work products to assure

s conformity with project requirements.

SUP.8 Perform problem resolution Analyze all problems detected and assure their removal; perform preventive actions in cases of identified recurrent problems.

MAN.1 Manage the project Prepare project plan, including required resources; coordinate and manage project to produce the required project products.

MAN.2 Manage quality Define quality of project products and/or services to fulfill quality requirements and assure customer satisfaction.

MAN.3 Manage risk Perform periodic risk surveys to detect risks, analyze their expected impact and carry out necessary actions to eliminate risks and reduce damages.

MAN.4 Manage subcontractors Select qualified subcontractors, assure adequate contract terms, control their performance and quality of products.

ORG.1 Engineer the business Establish an organizational environment that supports team members in their efforts; encourage professional achievements and improved effectiveness and efficiency.

ORG.2 Define the process Support teams’ performance by defining procedures, standards and library of reused software code and design modules.

ORG.3 Improve the process Invest constant effort to improve process effectiveness and efficiency and control implementation.

ORG.4 Provide skilled human resources Provide adequate training and instruction to assure that the organization’s human resources acquire required professional skills ORG.4 Provide skilled human resources Provide adequate training and instruction to assure that the organization’s human resources acquire required professional skills

SQA project process standards – IEEE software engineering standards

Chapter outline

24.1 Structure and content of IEEE software engineering standards 509

24.2 IEEE/EIA Std 12207 – software life cycle processes 510

24.2.1 Purpose 510

24.2.2 The 12207 software life cycle architecture 511

24.2.3 Underlying concepts 512

24.2.4 Contents 514

24.3 IEEE Std 1012 – verification and validation 514

24.3.1 Purpose 514

24.3.2 Underlying concepts 514

24.3.3 The standard’s content 516

24.4 IEEE Std 1028 – reviews 519

24.4.1 Purpose 519

24.4.2 Underlying concepts 519

24.4.3 The standard’s content 520 Summary

521 Selected bibliography

524 Review questions

524 Topics for discussion

525 Appendix 24A: IEEE Software Engineering Standards

526 Appendix 24B: MIL-STD-498: list of Data Item Descriptions (DIDs)

Appendix 24C: Task structure for a primary process according to IEEE/EIA Std 12207 – example

Project process standards focus on methodologies for carrying out software development and maintenance – on the “how” of software development project implementation. A project process standard is devised by delineating

508 The main benefits gained by use of project process standards are:

24 ■ The ability to apply the most professional software development and S

maintenance methodologies available.

A project

The ability to apply state-of-the-art project process procedures.

Better mutual understanding and coordination among teams, especially between development and maintenance teams.

Greater cooperation between the software developer and external partic-

proc

ipants in the project.

es

Better understanding and cooperation between suppliers and customers,

based on incorporation of known standards within the contract.

andard st

To better acquaint the reader with the issues involved, the benefits of stan- dards use, the organizations involved in standards development and the ways in which standards contribute to SQA are discussed in the introduction to

s – Part VI. IEEE

Due to their comparable characteristics, many SQA project process stan- dards naturally operate as software engineering standards and vice versa.

sof

Many organizations – international standards organizations, national standards institutes, professional organizations and industry organizations,

tw

among others – are occupied in developing, adapting and enforcing SQA

are engineering st

project process standards. A 1997 survey quoted by Moore (1999) lists 315 software standards developed by 46 different organizations. There is a grow- ing tendency among these organizations to abandon local standards and join in efforts to create international standards, as discussed in the introduction to Part VI of this book.

One of the organizations leading this trend is the American Institute of Electrical and Electronics Engineers (IEEE), and the associated IEEE

andard

Computer Society. One of IEEE’s main contributions lies in the generation, promulgation and promotion of standards use. A subgroup of the IEEE standards working group was formed in 1976 to develop SQA standards,

later published as the IEEE Software Engineering Standard Collection (in regularly updated editions). These standards, most of which may be classi- fied as project process standards, become a major source for international standards. For this reason – as well as space limitations – this chapter is devoted to IEEE software engineering standards, a small sample of which will be reviewed. The complete list of standards available at the time of writ- ing is presented in Appendix 24A.

For discussions of the directions taken by SQA standardization and the range of applications, see Schulmeyer (1999) and Heil (1999), respectively. The chapter deals with the following topics:

After completing this chapter, you will be able to: 509

Explain the concepts embodied in IEEE/EIA Std 12207.

Explain the concepts embodied in IEEE Std 1012.

Explain the essence of the SVVP as required by IEEE Std 1012.

Struct

Explain the concepts embodied in IEEE Std 1028.

ure and c