Testing for Real-Time Systems
17.7.4 Testing for Real-Time Systems
The time-dependent, asynchronous nature of many real-time applications adds a new and potentially difficult element to the testing mix—time. Not only does the test case designer have to consider white- and black-box test cases but also event handling (i.e., interrupt processing), the timing of the data, and the parallelism of the tasks (processes) that handle the data. In many situations, test data provided when a real- The time-dependent, asynchronous nature of many real-time applications adds a new and potentially difficult element to the testing mix—time. Not only does the test case designer have to consider white- and black-box test cases but also event handling (i.e., interrupt processing), the timing of the data, and the parallelism of the tasks (processes) that handle the data. In many situations, test data provided when a real-
For example, the real-time software that controls a new photocopier accepts oper- ator interrupts (i.e., the machine operator hits control keys such as RESET or DARKEN) with no error when the machine is making copies (in the "copying" state). These same operator interrupts, if input when the machine is in the "jammed" state, cause a dis- play of the diagnostic code indicating the location of the jam to be lost (an error).
Real-Time Systems In addition, the intimate relationship that exists between real-time software and
its hardware environment can also cause testing problems. Software tests must con- sider the impact of hardware faults on software processing. Such faults can be extremely difficult to simulate realistically.
Comprehensive test case design methods for real-time systems have yet to evolve. However, an overall four-step strategy can be proposed:
Task testing. The first step in the testing of real-time software is to test each task independently. That is, white-box and black-box tests are designed and executed for each task. Each task is executed independently during these tests. Task testing uncovers errors in logic and function but not timing or behavior.
Behavioral testing. Using system models created with CASE tools, it is pos- sible to simulate the behavior of a real-time system and examine its behavior as a consequence of external events. These analysis activities can serve as
the basis for the design of test cases that are conducted when the real-time The Software Testing
WebRef
software has been built. Using a technique that is similar to equivalence par- Discussion Forum
presents topics of interest titioning (Section 17.6.1), events (e.g., interrupts, control signals) are catego- to testing professionals:
rized for testing. For example, events for the photocopier might be user
www.ondaweb.com
interrupts (e.g., reset counter), mechanical interrupts (e.g., paper jammed),
/HyperNews/ get.cgi/forums/
system interrupts (e.g., toner low), and failure modes (e.g., roller overheated).
sti.html
Each of these events is tested individually and the behavior of the executable system is examined to detect errors that occur as a consequence of process- ing associated with these events. The behavior of the system model (devel- oped during the analysis activity) and the executable software can be compared for conformance. Once each class of events has been tested, events are presented to the system in random order and with random fre- quency. The behavior of the software is examined to detect behavior errors.
Intertask testing. Once errors in individual tasks and in system behavior have been isolated, testing shifts to time-related errors. Asynchronous tasks that are known to communicate with one another are tested with different data rates and processing load to determine if intertask synchronization errors will occur. In addition, tasks that communicate via a message queue or data store are tested to uncover errors in the sizing of these data storage areas.
PA R T T H R E E C O N V E N T I O N A L M E T H O D S F O R S O F T WA R E E N G I N E E R I N G
System testing. Software and hardware are integrated and a full range of system tests (Chapter 18) are conducted in an attempt to uncover errors at the software/hardware interface. Most real-time systems process interrupts. Therefore, testing the handling of these Boolean events is essential. Using the state transition diagram and the control specification (Chapter 12), the tester develops a list of all possible interrupts and the processing that occurs as a consequence of the interrupts. Tests are then designed to assess the fol- lowing system characteristics:
• Are interrupt priorities properly assigned and properly handled? • Is processing for each interrupt handled correctly? • Does the performance (e.g., processing time) of each interrupt-handling
procedure conform to requirements? • Does a high volume of interrupts arriving at critical times create prob-
lems in function or performance? In addition, global data areas that are used to transfer information as part of inter-
rupt processing should be tested to assess the potential for the generation of side effects.
Parts
» The Concurrent Development Model
» SUMMARY Software engineering is a discipline that integrates process, methods, and tools for
» PEOPLE In a study published by the IEEE [CUR88], the engineering vice presidents of three
» THE PROCESS The generic phases that characterize the software process—definition, development,
» THE PROJECT In order to manage a successful software project, we must understand what can go
» METRICS IN THE PROCESS AND PROJECT DOMAINS
» Extended Function Point Metrics
» METRICS FOR SOFTWARE QUALITY
» INTEGRATING METRICS WITHIN THE SOFTWARE PROCESS
» METRICS FOR SMALL ORGANIZATIONS
» ESTABLISHING A SOFTWARE METRICS PROGRAM
» Obtaining Information Necessary for Scope
» An Example of LOC-Based Estimation
» QUALITY CONCEPTS 1 It has been said that no two snowflakes are alike. Certainly when we watch snow
» SUMMARY Software quality assurance is an umbrella activity that is applied at each step in the
» R diagram 1.4 <part-of> data model; data model <part-of> design specification;
» SYSTEM MODELING Every computer-based system can be modeled as an information transform using an
» Facilitated Application Specification Techniques
» Data Objects, Attributes, and Relationships
» Entity/Relationship Diagrams
» Hatley and Pirbhai Extensions
» Creating an Entity/Relationship Diagram
» SUMMARY Design is the technical kernel of software engineering. During design, progressive
» Data Modeling, Data Structures, Databases, and the Data Warehouse
» Data Design at the Component Level
» A Brief Taxonomy of Styles and Patterns
» Quantitative Guidance for Architectural Design
» Isolate the transform center by specifying incoming and outgoing
» SUMMARY Software architecture provides a holistic view of the system to be built. It depicts the
» The User Interface Design Process
» Defining Interface Objects and Actions
» D E S I G N E VA L U AT I O N
» Testing for Real-Time Systems
» Organizing for Software Testing
» Criteria for Completion of Testing
» The Transition to a Quantitative View
» The Attributes of Effective Software Metrics
» Architectural Design Metrics
» Component-Level Design Metrics
» SUMMARY Software metrics provide a quantitative way to assess the quality of internal product
» Encapsulation, Inheritance, and Polymorphism
» Identifying Classes and Objects
» The Common Process Framework for OO
» OO Project Metrics and Estimation
» Event Identification with Use-Cases
» SUMMARY Object-oriented analysis methods enable a software engineer to model a problem by
» Partitioning the Analysis Model
» Designing Algorithms and Data Structures
» Program Components and Interfaces
» SUMMARY Object-oriented design translates the OOA model of the real world into an
» Testing Surface Structure and Deep Structure
» Deficiencies of Less Formal Approaches 1
» What Makes Cleanroom Different?
» Design Refinement and Verification
» SUMMARY Cleanroom software engineering is a formal approach to software development that
» Structural Modeling and Structure Points
» Describing Reusable Components
» SUMMARY Component-based software engineering offers inherent benefits in software quality,
» Guidelines for Distributing Application Subsystems
» Middleware and Object Request Broker Architectures
» An Overview of a Design Approach
» Consider expert Web developer will create a complete design, but time and cost can be appropriate
» A Software Reengineering Process Model
» Reverse Engineering to Understand Data
» Forward Engineering for Client/Server Architectures
» SUMMARY Reengineering occurs at two different levels of abstraction. At the business level,
Show more