THE QUALITY MOVEMENT Today, senior managers at companies throughout the industrialized world recognize

8.2 THE QUALITY MOVEMENT Today, senior managers at companies throughout the industrialized world recognize

that high product quality translates to cost savings and an improved bottom line. However, this was not always the case. The quality movement began in the 1940s with the seminal work of W. Edwards Deming [DEM86] and had its first true test in Japan. Using Deming’s ideas as a cornerstone, the Japanese developed a systematic that high product quality translates to cost savings and an improved bottom line. However, this was not always the case. The quality movement began in the 1940s with the seminal work of W. Edwards Deming [DEM86] and had its first true test in Japan. Using Deming’s ideas as a cornerstone, the Japanese developed a systematic

such as “total quality management” (TQM). 2 Although terminology differs across dif- TQM can be applied to ferent companies and authors, a basic four step progression is normally encountered

computer software. The TQM approach

and forms the foundation of any good TQM program. stresses continuous

The first step, called kaizen, refers to a system of continuous process improvement. process improvement.

The goal of kaizen is to develop a process (in this case, the software process) that is visible, repeatable, and measurable.

The second step, invoked only after kaizen has been achieved, is called atarimae hinshitsu. This step examines intangibles that affect the process and works to opti- mize their impact on the process. For example, the software process may be affected by high staff turnover, which itself is caused by constant reorganization within a com- pany. Maybe a stable organizational structure could do much to improve the quality of software. Atarimae hinshitsu would lead management to suggest changes in the way reorganization occurs.

While the first two steps focus on the process, the next step, called kansei (trans-

WebRef

lated as “the five senses”), concentrates on the user of the product (in this case, soft-

A wide variety of ware). In essence, by examining the way the user applies the product kansei leads to resources for continuous

improvement in the product itself and, potentially, to the process that created it. process improvement and

TQM can be found at Finally, a step called miryokuteki hinshitsu broadens management concern beyond

deming.eng.clemson.

the immediate product. This is a business-oriented step that looks for opportunity in

edu/

related areas identified by observing the use of the product in the marketplace. In the software world, miryokuteki hinshitsu might be viewed as an attempt to uncover new and profitable products or applications that are an outgrowth from an existing computer-based system.

For most companies kaizen should be of immediate concern. Until a mature soft- ware process (Chapter 2) has been achieved, there is little point in moving to the next steps.

8 . 3 S O F T WA R E Q U A L I T Y A S S U R A N C E

Even the most jaded software developers will agree that high-quality software is an important goal. But how do we define quality? A wag once said, "Every program does something right, it just may not be the thing that we want it to do."

Many definitions of software quality have been proposed in the literature. For our purposes, software quality is defined as

? How do we

define

Conformance to explicitly stated functional and performance requirements, explicitly doc-

software quality? umented development standards, and implicit characteristics that are expected of all pro-

fessionally developed software.

2 See [ART92] for a comprehensive discussion of TQM and its use in a software context and [KAP95] for a discussion of the use of the Baldrige Award criteria in the software world.

There is little question that this definition could be modified or extended. In fact, a definitive definition of software quality could be debated endlessly. For the purposes of this book, the definition serves to emphasize three important points:

1. Software requirements are the foundation from which quality is measured. Lack of conformance to requirements is lack of quality.

2. Specified standards define a set of development criteria that guide the man- ner in which software is engineered. If the criteria are not followed, lack of quality will almost surely result.

3. A set of implicit requirements often goes unmentioned (e.g., the desire for ease of use and good maintainability). If software conforms to its explicit requirements but fails to meet implicit requirements, software quality is sus- pect.

8.3.1 Background Issues

Quality assurance is an essential activity for any business that produces products to

be used by others. Prior to the twentieth century, quality assurance was the sole responsibility of the craftsperson who built a product. The first formal quality assur-

“We made too many ance and control function was introduced at Bell Labs in 1916 and spread rapidly wrong mistakes.”

throughout the manufacturing world. During the 1940s, more formal approaches to

Yogi Berra

quality control were suggested. These relied on measurement and continuous process improvement as key elements of quality management.

Today, every company has mechanisms to ensure quality in its products. In fact, explicit statements of a company's concern for quality have become a marketing ploy during the past few decades.

The history of quality assurance in software development parallels the history of quality in hardware manufacturing. During the early days of computing (1950s and

WebRef

1960s), quality was the sole responsibility of the programmer. Standards for quality An in-depth tutorial and

assurance for software were introduced in military contract software development wide-ranging resources for

during the 1970s and have spread rapidly into software development in the com- quality management can

be found at mercial world [IEE94]. Extending the definition presented earlier, software quality

www.management.

assurance is a "planned and systematic pattern of actions" [SCH98] that are required

gov.

to ensure high quality in software. The scope of quality assurance responsibility might best be characterized by paraphrasing a once-popular automobile commercial: "Qual- ity Is Job #1." The implication for software is that many different constituencies have software quality assurance responsibility—software engineers, project managers, customers, salespeople, and the individuals who serve within an SQA group.

The SQA group serves as the customer's in-house representative. That is, the peo- ple who perform SQA must look at the software from the customer's point of view. Does the software adequately meet the quality factors noted in Chapter 19? Has soft- The SQA group serves as the customer's in-house representative. That is, the peo- ple who perform SQA must look at the software from the customer's point of view. Does the software adequately meet the quality factors noted in Chapter 19? Has soft-

8.3.2 SQA Activities

Software quality assurance is composed of a variety of tasks associated with two dif- ferent constituencies—the software engineers who do technical work and an SQA group that has responsibility for quality assurance planning, oversight, record keep- ing, analysis, and reporting.

Software engineers address quality (and perform quality assurance and quality control activities) by applying solid technical methods and measures, conducting for- mal technical reviews, and performing well-planned software testing. Only reviews are discussed in this chapter. Technology topics are discussed in Parts Three through Five of this book.

The charter of the SQA group is to assist the software team in achieving a high- quality end product. The Software Engineering Institute [PAU93] recommends a set of SQA activities that address quality assurance planning, oversight, record keeping, analysis, and reporting. These activities are performed (or facilitated) by an inde- pendent SQA group that:

? Prepares an SQA plan for a project. The plan is developed during project plan-

What is the

role of an

ning and is reviewed by all interested parties. Quality assurance activities performed

SQA group?

by the software engineering team and the SQA group are governed by the plan. The plan identifies

• evaluations to be performed • audits and reviews to be performed • standards that are applicable to the project • procedures for error reporting and tracking • documents to be produced by the SQA group • amount of feedback provided to the software project team

Participates in the development of the project’s software process descrip-

tion. The software team selects a process for the work to be performed. The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.

Reviews software engineering activities to verify compliance with the defined

software process. The SQA group identifies, documents, and tracks deviations from the process and verifies that corrections have been made.

Audits designated software work products to verify compliance with those

defined as part of the software process. The SQA group reviews selected work products; identifies, documents, and tracks deviations; verifies that corrections have been made; and periodically reports the results of its work to the project manager.

Ensures that deviations in software work and work products are documented

and handled according to a documented procedure. Deviations may be encoun- tered in the project plan, process description, applicable standards, or technical work products.

Records any noncompliance and reports to senior management. Noncom- pliance items are tracked until they are resolved.

In addition to these activities, the SQA group coordinates the control and manage- ment of change (Chapter 9) and helps to collect and analyze software metrics.

8 . 4 S O F T WA R E R E V I E W S

Software reviews are a "filter" for the software engineering process. That is, reviews are applied at various points during software development and serve to uncover errors

Like water filters, FTRs and defects that can then be removed. Software reviews "purify" the software engi- tend to retard the

neering activities that we have called analysis, design, and coding. Freedman and “flow” of software

Weinberg [FRE90] discuss the need for reviews this way: engineering activities.

Too few and the flow

Technical work needs reviewing for the same reason that pencils need erasers: To err is

is “dirty.” Too many

human. The second reason we need technical reviews is that although people are good at

and the flow slows to

catching some of their own errors, large classes of errors escape the originator more eas-

a trickle. Use metrics

ily than they escape anyone else. The review process is, therefore, the answer to the prayer

to determine which reviews work and

of Robert Burns:

which may not be effective. Take the

O wad some power the giftie give us

ineffective ones out of

to see ourselves as other see us

the flow.

A review—any review—is a way of using the diversity of a group of people to: 1. Point out needed improvements in the product of a single person or team;

2. Confirm those parts of a product in which improvement is either not desired or not needed;

3. Achieve technical work of more uniform, or at least more predictable, quality than can be achieved without reviews, in order to make technical work more manageable.

Many different types of reviews can be conducted as part of software engineer- ing. Each has its place. An informal meeting around the coffee machine is a form of review, if technical problems are discussed. A formal presentation of software design to an audience of customers, management, and technical staff is also a form of Many different types of reviews can be conducted as part of software engineer- ing. Each has its place. An informal meeting around the coffee machine is a form of review, if technical problems are discussed. A formal presentation of software design to an audience of customers, management, and technical staff is also a form of

8.4.1 Cost Impact of Software Defects

The IEEE Standard Dictionary of Electrical and Electronics Terms (IEEE Standard 100- 1992) defines a defect as “a product anomaly.” The definition for fault in the hardware context can be found in IEEE Standard 610.12-1990:

(a) A defect in a hardware device or component; for example, a short circuit or broken wire. (b) An incorrect step, process, or data definition in a computer program. Note: This definition is used primarily by the fault tolerance discipline. In common usage, the terms "error" and "bug" are used to express this meaning. See also: data-sensitive fault; program- sensitive fault; equivalent faults; fault masking; intermittent fault.

Within the context of the software process, the terms defect and fault are synony- mous. Both imply a quality problem that is discovered after the software has been released to end-users (or to another activity in the software process). In earlier chap- ters, we used the term error to depict a quality problem that is discovered by software engineers (or others) before the software is released to the end-user (or to another activity in the software process).

The primary objective of formal technical reviews is to find errors during the process so that they do not become defects after release of the software. The obvious bene- fit of formal technical reviews is the early discovery of errors so that they do not prop- agate to the next step in the software process.

The primary objective

A number of industry studies (by TRW, Nippon Electric, Mitre Corp., among oth- of an FTR is to find

errors before they are ers) indicate that design activities introduce between 50 and 65 percent of all errors passed on to another

(and ultimately, all defects) during the software process. However, formal review tech- software engineering

niques have been shown to be up to 75 percent effective [JON86] in uncovering design activity or released to

flaws. By detecting and removing a large percentage of these errors, the review process the customer.

substantially reduces the cost of subsequent steps in the development and support phases.

To illustrate the cost impact of early error detection, we consider a series of rela- tive costs that are based on actual cost data collected for large software projects [IBM81]. 3 Assume that an error uncovered during design will cost 1.0 monetary unit to correct. Relative to this cost, the same error uncovered just before testing com- mences will cost 6.5 units; during testing, 15 units; and after release, between 60 and 100 units.

3 Although these data are more than 20 years old, they remain applicable in a modern context.

Development step

Errors passed through

Errors from

Percent

previous step

Amplified errors 1 : x

efficiency

Errors passed

for error

to next step

detection Newly generated errors

8.4.2 Defect Amplification and Removal

A defect amplification model [IBM81] can be used to illustrate the generation and detection of errors during the preliminary design, detail design, and coding steps of the software engineering process. The model is illustrated schematically in Fig-

“Some maladies, as ure 8.2. A box represents a software development step. During the step, errors may doctors say, at their

be inadvertently generated. Review may fail to uncover newly generated errors and beginning are easy

to cure but difficult errors from previous steps, resulting in some number of errors that are passed through. to recognize . . . but

In some cases, errors passed through from previous steps are amplified (amplifica- in the course of time

tion factor, x) by current work. The box subdivisions represent each of these charac- when they have not

teristics and the percent of efficiency for detecting errors, a function of the thoroughness at first been

recognized and of the review. treated, become

Figure 8.3 illustrates a hypothetical example of defect amplification for a software easy to recognize

development process in which no reviews are conducted. Referring to the figure, each but difficult to cure.”

test step is assumed to uncover and correct 50 percent of all incoming errors with-

Niccolo Machiavelli

out introducing any new errors (an optimistic assumption). Ten preliminary design defects are amplified to 94 errors before testing commences. Twelve latent errors are released to the field. Figure 8.4 considers the same conditions except that design and code reviews are conducted as part of each development step. In this case, ten ini- tial preliminary design errors are amplified to 24 errors before testing commences. Only three latent errors exist. Recalling the relative costs associated with the dis- covery and correction of errors, overall cost (with and without review for our hypo- thetical example) can be established. The number of errors uncovered during each of the steps noted in Figures 8.3 and 8.4 is multiplied by the cost to remove an error (1.5 cost units for design, 6.5 cost units before test, 15 cost units during test, and 67 cost units after release). Using these data, the total cost for development and main- tenance when reviews are conducted is 783 cost units. When no reviews are con- ducted, total cost is 2177 units—nearly three times more costly.

To conduct reviews, a software engineer must expend time and effort and the development organization must spend money. However, the results of the preceding example leave little doubt that we can pay now or pay much more later. Formal tech-

Preliminary design

Defect

amplification,

Detail design

no reviews

0 0% 10 6 6 Code/unit test

94 Integration test

Validation test

To integration System test

Latent errors

F I G U R E 8.4

Preliminary design

Defect amplification,

0 Detail design

2 Code/unit test

24 Integration test

Validation test

0 50% 12 System test To integration

Latent errors nical reviews (for design and other technical activities) provide a demonstrable cost

benefit. They should be conducted.