Running Error Analysis

2.5.1 Running Error Analysis

A different approach to rounding error analysis is to perform the analysis automatically, for each particular computation . This gives an a posteriori error analysis in contrast to the a priori error analysis discussed above.

A simple form of a posteriori analysis, called running error analysis, was used in the early days of computing; see Wilkinson [380]. To illustrate his idea we rewrite the basic model for floating-point arithmetic as

x op y = f l (x op y)(1 + ǫ).

This holds for most implementations of floating-point arithmetic. Then, the actual error can be estimated |f l (x op y) − x op y| ≤ u|f l (x op y)|. Note that the error is now given

in terms of the computed result and is available in the computer at the time the operation is performed. This running error analysis can often be easily implemented. We just take an existing program and modify it, so that as each arithmetic operation is performed, the absolute value of the computed results is added into the accumulating error bound.

Example 2.5.1.

The inner product f l (x T y) is computed by the program

s = 0; η = 0; for i = 1, 2, . . . , n

t = f l (x i y i ) ; η = η + |t|; s = f l (s + t); η = η + |s|;

end

For the final error we have the estimate |f l (x T y) −x T y | ≤ ηu. Note that a running error analysis takes advantage of cancellations in the sum. This is in contrast to the previous

estimates, which we call a priori error analysis, where the error estimate is the same for all distribution of signs of the elements x i and y i .

Efforts have been made to design the computational unit of a computer so that it gives, in every arithmetic operation, only those digits of the result which are judged to be significant (possibly with a fixed number of extra digits), so-called unnormalized floating arithmetic . This method reveals poor construction in algorithms, but in many other cases it gives a significant and unnecessary loss of accuracy. The mechanization of the rules, which

a knowledgeable and experienced person would use for control of accuracy in hand calcu- lation, is not as free from problems as one might expect. As a complement to arithmetical operations of conventional type, the above type of arithmetic is of some interest, but it is doubtful that it will ever be widely used.

A fundamental difficulty in automatic control of accuracy is that to decide how many digits are needed in a quantity to be used in later computation, one needs to consider the entire

146Chapter 2. How to Obtain and Estimate Accuracy context of the computations . It can in fact occur that the errors in many operands depend on

each other in such a way that they cancel each other. Such a cancellation of error, which is a completely different phenomenon from the previously discussed cancellation of terms, is most common in larger problems, but will be illustrated here with a simple example.

Example 2.5.2.

√ Suppose we want to compute y = z 1 +z 2 , where z 1 = x 2 + 1, z 2 = 200 − x, x = 100 ± 1, with a rounding error which is negligible compared to that resulting from

the errors in z 1 and z 2 . The best possible error bounds in the intermediate results are z 1 = 100 ± 1, z 2 = 100 ± 1. It is then tempting to be satisfied with the result y = 200 ± 2. But the errors in z 1 and z 2 due to the uncertainty in x will, to a large extent, cancel each other! This becomes clear if we rewrite the expression as

an absolute error less than about 2/(200) 2 = 0.5 · 10 −4 . Therefore, using the expression y =z 1 +z 2 the intermediate results z 1 and z 2 should be computed to four decimals even though the last integer in these is uncertain. The result is y = 200.0050 ± 1

2 10 −4 . In larger problems, such a cancellation of errors can occur even though one cannot

easily give a way to rewrite the expressions involved. The authors have seen examples where the final result, a sum of seven terms, was obtained correctly to eight decimals even though the terms, which were complicated functions of the solution to a system of nonlinear equations with 14 unknowns, were correct only to three decimals! Another nontrivial example is given in Example 2.4.12.