Software quality

24.1 Software quality

The fundamentals of quality management were established by manufacturing indus- try in a drive to improve the quality of the products that were being made. As part of this, they developed a definition of ‘quality’, which was based on conformance with a detailed product specification (Crosby, 1979) and the notion of tolerances. The underlying assumption was that products could be completely specified and proce- dures could be established that could check a manufactured product against its spec- ification. Of course, products will never exactly meet a specification so some tolerance was allowed. If the product was ‘almost right’, it was classed as acceptable.

Software quality is not directly comparable with quality in manufacturing. The idea of tolerances is not applicable to digital systems and, for the following reasons, it may be impossible to come to an objective conclusion about whether or not a soft- ware system meets its specification:

1. As I discussed in Chapter 4, which covered requirements engineering, it is diffi- cult to write complete and unambiguous software specifications. Software developers and customers may interpret the requirements in different ways and it may be impossible to reach agreement on whether or not software conforms to its specification.

2. Specifications usually integrate requirements from several classes of stakehold- ers. These requirements are inevitably a compromise and may not include the requirements of all stakeholder groups. The excluded stakeholders may there- fore perceive the system as a poor quality system, even though it implements the agreed requirements.

3. It is impossible to measure certain quality characteristics (e.g., maintainability) directly and so they cannot be specified in an unambiguous way. I discuss the difficulties of measurement in Section 24.4.

Because of these problems, the assessment of software quality is a subjective process where the quality management team has to use their judgment to decide if an acceptable level of quality has been achieved. The quality management team has to consider whether or not the software is fit for its intended purpose. This involves answering questions about the system’s characteristics. For example:

1. Have programming and documentation standards been followed in the develop- ment process?

2. Has the software been properly tested?

3. Is the software sufficiently dependable to be put into use?

4. Is the performance of the software acceptable for normal use?

656 Chapter 24 ■ Quality management

Efficiency Figure 24.2 Software

Resilience

Modularity

Learnability quality attributes

Robustness

Complexity

5. Is the software usable?

6. Is the software well structured and understandable? There is a general assumption in software quality management that the system

will be tested against its requirements. The judgment on whether or not it delivers the required functionality should be based on the results of these tests. Therefore, the QA team should review the tests that have been developed and examine the test records to check that testing has been properly carried out. In some organizations, the quality management team is responsible for system testing but, sometimes, a sep- arate system testing group is made responsible for this.

The subjective quality of a software system is largely based on its non-functional characteristics. This reflects practical user experience—if the software’s functional- ity is not what is expected, then users will often just work around this and find other ways to do what they want to do. However, if the software is unreliable or too slow, then it is practically impossible for them to achieve their goals.

Therefore software quality is not just about whether the software functionality has been correctly implemented, but also depends on non-functional system attrib- utes. Boehm, et al. (1978) suggested that there were 15 important software quality attributes, as shown in Figure 24.2. These attributes relate to the software depend- ability, usability, efficiency, and maintainability. As I have discussed in Chapter 11, I believe that dependability attributes are usually the most important quality attributes of a system. However, the software’s performance is also very important. Users will reject software that is too slow.

It is not possible for any system to be optimized for all of these attributes—for example, improving robustness may lead to loss of performance. The quality plan should therefore define the most important quality attributes for the software that is being developed. It may be that efficiency is critical and other factors have to be sacri- ficed to achieve this. If you have stated this in the quality plan, the engineers working on the development can cooperate to achieve this. The plan should also include a defi- nition of the quality assessment process. This should be an agreed way of assessing whether some quality, such as maintainability or robustness, is present in the product.

An assumption that underlies software quality management is that the quality of software is directly related to the quality of the software development process. This again comes from manufacturing systems where product quality is intimately related

24.2 ■ Software standards 657

Define Process

Develop

Assess Product

Yes Standardize

Process Figure 24.3 Process-

Process

OK

based quality

to the production process. A manufacturing process involves configuring, setting up, and operating the machines involved in the process. Once the machines are operating correctly, product quality naturally follows. You measure the quality of the product and change the process until you achieve the quality level that you need. Figure 24.3 illustrates this process-based approach to achieving product quality.

There is a clear link between process and product quality in manufacturing because the process is relatively easy to standardize and monitor. Once manufactur- ing systems are calibrated, they can be run again and again to output high-quality products. However, software is not manufactured—it is designed. In software devel- opment, therefore, the relationship between process quality and product quality is more complex. Software development is a creative rather than a mechanical process, so the influence of individual skills and experience is significant. External factors, such as the novelty of an application or commercial pressure for an early product release, also affect product quality irrespective of the process used.

There is no doubt that the development process used has a significant influence on the quality of the software and that good processes are more likely to lead to good quality software. Process quality management and improvement can lead to fewer defects in the software being developed. However, it is difficult to assess software quality attributes, such as maintainability, without using the software for a long period. Consequently, it is hard to tell how process characteristics influence these attributes. Furthermore, because of the role of design and creativity in the software process, process standardization can sometimes stifle creativity, which leads to poorer rather than better quality software.