Testing Verification, validation and testing

142 Chapter 6 n Software development n Recovery testing. Because your system may be handling inconsistent or error-prone data it may be important that your system is robust and does not crash when it encounters such problems. Recovery testing involves testing how well the program recovers from ‘bad’ data, an overflow, a crash, etc. You can test your system in this way by generating ‘bad’ data that can be used during testing. n Documentation testing. Your clientuser may have requested comprehensive documentation to support your software and this may be assessed as part of your overall project mark. Documentation testing involves evaluating how good the supporting documentation is for your system. This can be performed by users, non- expert assessors naïve testers, the client, yourself and experts. It can be performed off-line i.e., assessed without the software system to hand, in which case you will be checking for things like readability, layout and clarity or on-line assessed with the program to hand. During on-line testing you will be assessing how well the documentation supports the user in using the system: Do any tutorials match the program?; Is help available if things go wrong?; etc. n Interface testing. This is particularly important in web development projects in which the user can be very critical of the overall look and feel of the site and has many other web sites with which to compare it. The home page of a web site is effectively the site as far as the user is concerned so getting this right is important. Interface testing also checks whether the system is usable from a functional perspective. The system you have developed may perform some quite complex, interesting functions, but if the interface is clumsy, difficult to use, or not intuitive it may well put the user off. You should consider reading guidelines on HCI Human-Computer Interaction principles when designing your system to ensure it is user-friendly and accessible. •

6.13 Quality

6.13.1 Introduction

In Chapter 4 we introduced a generic model of the project process and showed that any product produced by a project will have a certain level of scope and quality see Figure 4.1. Although scope what the product does, its functionality, capability, etc. has been defined separately, it is sometimes subsumed within the concept of quality – being one of the aspects that define a quality product. In student projects quality can refer to a number of things – the quality of the report content and presentation, the quality of the underlying research, the quality of the contribution the project has made, the quality of a software system devel- oped, etc. While the quality of your project is assessed by the examiners and measured by the mark you receive, in this section we focus on software quality and how software devel- opment projects control and manage quality. To begin this discussion we need to determine what is meant by software quality as it means different things to different people. 6.13.2 What is software quality? The first point to note is that a high-quality product might not necessarily be appropriate for our needs. For example, a small system with limited functionality may provide just the functionality a user needs, while a more complex system with lots of ‘bells and whistles’, although doing what a user requires, might be so complicated as to make it unworkable. 6.13 Quality 143 It is generally accepted that an appropriate definition of software quality is fitness for purpose. This does not mean the product has to be of high quality. When a customer buys a product they are selecting that product based on a number of quality criteria. Some of these criteria will be more important to some customers than others – some criteria may, for example, be of no concern at all. n Product quality. This is probably the most important quality characteristic that buy- ers will focus on. In terms of product quality the customer will be concerned with the product’s fitness for purpose – does it do what I need it to do? Does the product provide any additional functionality that I would like in addition to my needs? How reliable is the product – will it last me or will it need lots of maintenance, sup- port, repairs, replacement, etc. including spare parts, batteries, etc.? Is the product environmentally friendly will it have any impact on the environment through its every day use and eventual disposal or replacement? n Company quality. This refers to a consumer’s confidence in the company. How well will they deal with any problems I have? What is their returns policy? Will I be able to get a refund or exchange a faulty or unsatisfactory product? What is the company’s reputation? n Business quality. These days consumers are very keen on knowing the environmental impact of the supplier. Is the company ethically sound is the product produced by slave labour overseas, for example? What has been the environmental and social impact of the manufacture of the product? n Value quality. What is the product worth? What am I willing to pay for it? Does it represent good value for money or does it seem expensive for what it is? If it is expensive, is there an alternative that might satisfy my needs?

6.13.3 Quality assurance and quality control

Quality assurance is about getting procedures in place before a system is developed and delivered to ensure the system is of acceptable quality. By getting an appropriate development process in place beforehand, and monitoring and controlling this pro- cess, you will be laying the foundations towards producing a reasonable system. Without an appropriate set of processes in place, producing a system of reasonable quality will be subject to chance. Quality control is about measuring quality of the product as it is completed and en- suring it meets the quality standards defined. In Section 6.12 we looked at verification, Example Take the example of a top performance sports car worth many tens of thousands of pounds. You could argue that this was a high-quality car but in terms of fitness for purpose it might be sadly lacking. If you needed to pop to the shops or visit your friends, your top performance sports car may well be impractical small boot space so you could not buy much or carry a lot of luggage; high petrol consumption; expensive to insure, maintain; etc.. In terms of fitness for purpose, it would not be very good unless your purpose was to drive quickly and look good. 144 Chapter 6 n Software development validation and testing. These techniques are a type of quality control – checking that the system you have produced meets the requirements, works well and satisfies the client’s needs.

6.13.4 Quality standards

In developing commercial software systems, businesses invariably use and are certified to a quality standard of one kind or another. In this section we briefly discuss three of the most common quality standards that are used by organisations in product and partic- ularly software development. n ISO 9000. ISO 9000 is a series of standards for a quality management system. If a company is ISO 9000 certified, it means the company following an independent audit has a quality management system in place. This quality management system should, amongst other things, ensure that adequate records are kept of the business processes in terms of definition and monitoring and control, processes are contin- ually reviewed, and there is a quality control process in place. ISO 9000 includes a series of standards such as ISO 9001 which is aimed at companies that design, develop, manufacture, supply and maintain products and ISO 9004 which covers continual process improvement. Although not specifically aimed at software, the ISO 9000 series of standards is often adopted by software development companies to improve their performance and marketability. n ISO 9126. Introduced by the International Organisation for Standardization ISO in 1991, ISO 9126 defines six quality characteristics for software. These include: functionality, reliability, usability, efficiency, maintainability and portability. These characteristics can be broken down further depending on the nature of the system being developed for example, reliability can be broken down into fault tolerance, recoverability, etc.. By applying numerical ratings to each of the characteristics using Likert scales for example, it is possible to obtain relative measures of quality between systems. n Capability Maturity Model CMM. Unlike the ISO 9000 series of standards, CMM was developed specifically for the software development industry. Developed by the Software Engineering Institute it focuses on software process improvement. Organisations are graded on a five point scale according to their software process maturity. The levels range from 1, referring to organisations that have no defined software process in place and develop systems more by luck than judgement, to 5, which applies to organisations that not only have a well defined process in place but also invest in process change management. Although these quality standards are aimed at organisations, some of the principles underlying these standards may be applicable to your own software development project. For example, the principles of ISO 9000 in which you should have effective and recordable development processes in place is good practice to follow. The quality characteristics of software defined by ISO 9126 can be adapted to your own system evaluation. The principles of CMM in which you manage your development process and look at ways of improving your processes are also worthwhile.