Perturbation Analysis for Linear Systems

2.4.3 Perturbation Analysis for Linear Systems

in the previous section we know that the condition number of the inverse mapping x =

Assume that the elements of the matrix A are given data and subject to perturbations δA . The perturbed solution x + δx satisfies the linear system

(A + δA)(x + δx) = y.

Subtracting Ax = y we obtain (A + δA)δx = −δAx. Assuming also that the matrix (A + δA) = A(I + A −1 δA) is nonsingular and solving for δx yields

(2.4.13) which is the basic identity for the analysis. Taking norms gives

δx = −(I + A −1 δA) −1 A −1 δAx,

Neglecting second order terms,

(2.4.14) This shows that κ(A) is also the condition number of x = A −1 y with respect to perturbations

in A. For any real, orthogonal matrix Q we have

κ 2 (Q)

2 for any orthogonal P and Q. It follows that

κ 2 (P AQ) =κ 2 (A),

2.4. Error Propagation 135 i.e., the condition number of a matrix A is invariant under orthogonal transformations. This

important fact is one reason why orthogonal transformations play a central role in numerical linear algebra.

How large may κ be before we consider the problem to be ill-conditioned? That depends on the accuracy of the data and the accuracy desired in the solution. If the data have a relative error of 10 −7 , then we can guarantee a (normwise) relative error in the solution ≤ 10 4 −3 if κ ≤ 0.5·10 . But to guarantee a (normwise) relative error in the solution ≤ 10 −6 we need to have κ ≤ 5.

Example 2.4.7.

The Hilbert matrix H n of order n with elements

1 ≤ i, j ≤ n, is a notable example of an ill-conditioned matrix. In Table 2.4.1 approximate condition

H n (i, j ) =h ij = 1/(i + j − 1),

numbers of Hilbert matrices of order ≤ 12, computed in IEEE double precision, are given. For n > 12 the Hilbert matrices are too ill-conditioned even for IEEE double precision!

From a result by G. Szegö (see Gautschi [147, p. 34]) it follows that

2 + 1) 4(n+1)

κ (H n )

2 ≈ 3.5n 15/4 √ ∼e ,

2 πn

i.e., the condition numbers grow exponentially with n. Although the severe ill-conditioning exhibited by the Hilbert matrices is rare, moderately ill-conditioned linear systems do occur regularly in many practical applications!

Table 2.4.1. Condition numbers of Hilbert matrices of order ≤ 12.

The normwise condition analysis in the previous section usually is satisfactory when the linear system is “well scaled.” If this is not the case, then a componentwise analysis may give sharper bounds. We first introduce some notations. The absolute values |A| and |b| of a matrix A and vector b are interpreted componentwise

|A| ij = (|a ij |),

|b| i = (|b i |).

The partial ordering “≤” for the absolute values of matrices |A|, |B| and vectors |b|, |c| is to be interpreted componentwise: 40

|A| ≤ |B| ⇐⇒ |a ij | ≤ |b ij |, |b| ≤ |c| ⇐⇒ |b i | ≤ |c i |.

40 Note that A ≤ B in other contexts means that B − A is positive semidefinite.

136Chapter 2. How to Obtain and Estimate Accuracy It follows easily that |AB| ≤ |A| |B| and a similar rule holds for matrix-vector multiplica-

tion. Taking absolute values in (2.4.13) gives componentwise error bounds for the corre- sponding perturbations in x,

|δx| ≤ |(I + A −1 δA) −1 | |A −1 |(|δA||x| + |δb|).

The matrix (I − |A −1

Assume now that we have componentwise bounds for the perturbations in A and b, say

(2.4.15) Neglecting second order terms in ω and using (2.4.15) gives −1 |(|δA||x| + |δb|) ≤ ω|A −1 |(|A| |x| + |b|).

|δA| ≤ ω|A|,

|δb| ≤ ω|b|.

(2.4.16) Taking norms in (2.4.16) we get

2 ). (2.4.17) The scalar quantity

(2.4.18) is called the Bauer–Skeel condition number of the matrix A.

κ |A| (A)

A different way to examine the sensitivity of various matrix problems is the differen- tiation of a parametrized matrix. Suppose that λ is a scalar and that A(λ) is a matrix with elements a ij (λ) that are differentiable functions of λ. Then by the derivative of the matrix A(λ) we mean the matrix

Many of the rules for differentiation of scalar functions are easily generalized to differenti- ation of matrices. For differentiating a product of two matrices there holds

[A(λ)B(λ)] = [A(λ)]B(λ) + A(λ) [B(λ)]. (2.4.20) dλ

dλ

dλ

Assuming that A −1 (λ) exists, using this rule on the identity A −1 (λ)A(λ) = I we obtain

[A(λ)] = 0, dλ

[A −1 (λ) ]A(λ) + A −1

dλ

or, solving for the derivative of the inverse,

d dλ [A −1

(λ) ] = −A −1 (λ)

dλ [A(λ)]A −1 (λ).

2.4. Error Propagation 137