ADDITIONAL COVERAGE CRITERIA FOR SYSTEM TESTING

10.14 ADDITIONAL COVERAGE CRITERIA FOR SYSTEM TESTING

We discussed two coverage criteria, namely, state coverage and state transition coverage, to select test cases from FSM and EFSM models of software systems in Sections 10.5 and 10.13. Those two criteria focused on sequences of events, possibly including internal events, occurring at PCOs. In this section, we explain some more coverage criteria in line with the concepts of functional testing.

The reader may recall from Chapter 9 on functional testing that we identify the domains of input and output variables and select test data based on special values from those domains. For example, if an output variable takes on a small number of discrete values, then test cases are designed to make the SUT produce all those output values. If an output variable takes on values from a contiguous range, test data are selected such that the SUT produces the external points and an interior point in the specified range.

In line with the above concept of functional testing, in the following, we explain some coverage criteria for event-driven systems modeled as FSMs or EFSMs. First, we identify the PCOs, also referred to as ports—points where a SUT interacts with the external world. Next, we apply the following coverage criteria to select test cases:

PCO Coverage : Select test cases such that the SUT receives an event at each input PCO and produces an event at each output PCO.

Sequences of Events at PCOs : Select test cases such that common sequences of inputs and outputs occur at the PCOs. By a common sequence we mean sequences commonly found in the uses of the system.

Events Occurring in Different Contexts : In many applications, a user pro- duces an event for the system by pressing a button, for example. Here,

a button represents an input PCO, and pressing the button represents an event. However, the semantics of pressing a button, that is, interpretations of events at a PCO, depend on data contexts used by the system. For a given context, test data are selected such that all events, both desired and undesired, occur in the context.

Inopportune Events : A system is expected to discard invalid, or erroneous, events. On the other hand, inopportune events are normal events which occur at an inappropriate time.

Example: Automated Teller Machine. Let us consider the user interface of an ATM system as shown in Figure 10.46. A user selects one of the transaction options and specifies a transaction amount using buttons B1 through B6. The meaning of a

314 CHAPTER 10 TEST GENERATION FROM FSM MODELS

B2

B1 Display

B5 B6 Cash card slot Receipt slot

Cash dispensing door Deposit envelope door

Figure 10.46 User interface of ATM.

Select an option

4 5 6 <= Fund transfer

Check balance => B5 B6 Cash card slot Receipt slot

Cash dispensing door Deposit envelope door

Figure 10.47 Binding of buttons with user options.

button changes as the message in the display area changes. For example, a user can choose a transaction option, such as Deposit or Withdraw, by pressing buttons B2 or B4, respectively, as shown in Figure 10.47. However, as shown in Figure 10.48, when it comes to selecting an amount, buttons B2 and B4 represent options $40 and $100, respectively. Moreover, all the buttons for the amount context shown in Figure 10.48 do not represent the same type of data. For example, buttons B1 through B5 represent discrete, integer values, whereas button B6 gives the user an option to specify other values. In the context shown in Figure 10.47, buttons B3 and B5 are undefined, and, thus, those are potential sources of undesirable (erroneous) events. Test cases must be selected to observe how the system responds to undefined events. The OK button produces a normal event while the user is entering a PIN after inserting the cash card. The OK event essentially tells the system that the user

10.15 SUMMARY

Select an amount

0 CANCEL B3 $ 60

B6 Cash card slot Receipt slot

Cash dispensing door Deposit envelope door

Figure 10.48 Binding of buttons with cash amount.

has completely entered a PIN. However, it may not be meaningful to press OK in the context shown in Figure 10.48. Therefore, pressing the OK button produces an inopportune event for the system to handle. Test cases need to be selected to consider inopportune events in addition to normal (valid) and abnormal (invalid or erroneous) events.