Reliability as an Economic Function

13.3.7 Reliability as an Economic Function

So far we have made several simplifying assumptions as we analyze the reliability of a software product; in this section, we tentatively challenge these assumptions and offer

a more refined definition of reliability that lifts these restrictive assumptions: • We have assumed that the stakeholders in the operation of a software product are

a monolithic community, with a common stake in the reliable behavior of the system. In reality, a system may have several different stakeholders, having widely varying stakes in its reliable operation. Hence reliability is best viewed, not as a property of the product, but rather as an attribute of a product and a stake- holder. We represent it, not by a scalar (the MTTF), but rather as a vector, which has one entry per relevant stakeholder.

• We have assumed that the specification is a monolith, which carries a unique stake for each stakeholder, when in fact typical specifications are aggregates of several sub-specifications, representing distinct requirements whose stakes for any given stakeholder may vary widely. Hence whereas in the previous section we talked about the cost of a system failure as an attribute of the system, in this section we consider the structure of a specification, and we associate dif- ferent costs to different sub-specification, for each stakeholder.

• We have assumed that failure is a Boolean condition, whereby an execution either fails or succeeds, when in fact failure is rather a composite event, where the same system may succeed with respect to some requirements but fail with respect to others. Hence in estimating probabilities of failure, we do not consider failure as a single event, but rather as different events, having possibly different probabilities of occurrence and carrying different stakes even for the same stake- holder (let alone for different stakeholders).

To take into account all these dimensions of heterogeneity, we consider the random variable FC(H), which represents, for stakeholder H, the cost per unit of time that she/

13.3 STOCHASTIC CLAIMS: FAILURE PROBABILITY 301

he stands to incur as a result of possible system failures (FC stands for failure cost), and we let MFC(H) be the mean of variable FC(H) over various instances of system oper- ation. To fix our ideas, we quantify MFC(H) in terms of dollars per hour of operation, which we abbreviate by $/h. With this measure, it is no longer necessary to distinguish between reliability (freedom from failure with respect to common requirements) and safety (freedom from failure with respect to high stakes requirements), since the mean failure cost takes into account the costs associated with all relevant requirements, ran- ging from low stakes requirements to high stakes requirements.

We consider a system whose community of stakeholders includes n members

H 1 ,H 2 ,H 3 ,…H n , and whose specification R is structured as the aggregate of several requirements, say

R=R 1 R 2 R 3 … R m

be the probabilities that the system fails to satisfy requirements R 1 ,R 2 ,R 3 ,…R m during a unitary operation time (say, 1 hour of opera-

and we let P = P 1 ,P 2 ,P 3 , …P m

tion time). If we let ST(H i ,R j ) be the stakes that stakeholder H i has in meeting require- ment R j , then the mean failure cost of stakeholder H i can be approximated by the following formula:

This formula is not an exact estimate of the mean failure cost but is an approxima- tion thereof; this stems from two reasons, both of which result from the fact that spe- cifications R 1 ,R 2 ,R 3 ,…R m are not orthogonal, but rather overlap:

• Costs are not additive: when we consider the costs associated with failure to sat- isfy two distinct requirements R i and R j , the same loss may be counted twice because the two specifications are not totally orthogonal, hence their failures rep- resent related events.

• Probabilities are not multiplicative: If we consider two distinct specification components R i and R j that are part of the system specification, failure with respect to R i and failure with respect to R j are not statistically independent because the same error may cause both events.

Hence strictly speaking, the formula above is best understood as an upper bound of the mean failure cost, rather than an exact estimate; nevertheless, we use it as a con- venient (easy to compute) approximation. We recast the formula of MFC given above in matrix form, by means of the following notations:

• We let MFC be the column vector that has one entry per stakeholder, such that MFC(H i ) represents the mean failure cost of stakeholder H i .

302 TEST OUTCOME ANALYSIS

• We let P be the column vector that has one more entry than there are specification components, such that P(R j ) represents the probability that the system fails to satisfy requirement R j during a unitary execution time (e.g., 1 hour) and the extra entry represents the probability that no requirement is violated during a unitary execution time.

• We let ST be the matrix that has as many rows as there are stakeholders and as many columns as there are specification components and such that ST(H i ,R j ) represents the loss that stakeholder H i incurs if requirement R j is violated; we con- sider an additional column that represents the event that no requirement is violated.

Then the formula of mean failure cost can be written in relational form as follows (where • represents matrix product):