Householder Reduction to Tridiagonal Form

9.4 Householder Reduction to Tridiagonal Form

It was mentioned before that similarity transformations can be used to transform an eigenvalue problem to a form that is easier to solve. The most desirable of the “easy” forms is, of course, the diagonal form that results from the Jacobi method. However,

the Jacobi method requires about 10n 3 to 20n 3 multiplications, so that the amount of computation increases very rapidly with n. We are generally better off by reducing the matrix to the tridiagonal form, which can be done in precisely n − 2 transformations by the Householder method. Once the tridiagonal form is achieved, we still have to extract the eigenvalues and the eigenvectors, but there are effective means of dealing with that, as we see in the next article.

Householder Matrix

Householder’s transformation utilizes the Householder matrix

uu T

Q=I−

(9.37) Note that uu T in Eq. (9.36) is the outer product; that is, a matrix with the elements

H= u

2 u= 2 |u|

ij =u i u j . Since Q is obviously symmetric (Q = Q), we can write

H I− H =I−2 H +

uu T

u (2H) u T

=I−2 H +

H 2 =I

which shows that Q is also orthogonal. Now let x be an arbitrary vector and consider the transformation Qx. Choosing

u = x + ke 1 (9.38) where

so that

Qx = x − u = −ke 1 = −k 0 0 · · · 0 (9.39) Hence the transformation eliminates all elements of x except the first one.

Householder Reduction of a Symmetric Matrix

Let us now apply the following transformation to a symmetric n × n matrix A:

Qx QA ′ Here x is represents the first column of A with the first element omitted, and A ′

P 1 A=

is simply A with its first row and column removed. The matrix Q of dimensions

(n − 1) × (n − 1) is constructed using Eqs. (9.36)–(9.38). Referring to Eq. (9.39), we see that the transformation reduces the first column of A to

The transformation

11 (Qx) T A←P 1 AP 1 = Qx QA ′

thus tridiagonalizes the first row as well as the first column of A. Here is a diagram of the transformation for a 4 × 4 matrix:

The second row and column of A are reduced next by applying the transformation to the 3 × 3 lower right portion of the matrix. This transformation can be expressed as

A←P 2 AP 2 , where now

In Eq. (9.42) I 2 is a 2 × 2 identity matrix and Q is a (n − 2) × (n − 2) matrix constructed by choosing for x the bottom n − 2 elements of the second column of A. It takes a total of n − 2 transformations with

i = 1, 2, . . . , n − 2

to attain the tridiagonal form.

It is wasteful to form P i and then carry out the matrix multiplication P i AP i . We note that

Q=A T I− ′

uu

H =A − H =A −vu where

=A ′ −vu T −uv T + 2guu T

(9.45) it can be easily verified that the transformation can be written as

w = v − gu

(9.46) which gives us the following computational procedure which is to be carried out with

be the (n − i) × (n − i) lower right-hand portion of A.

2. Let x = A i+1,i A i+2,i ··· A n,i (the column of length n − i just to the left of

A ′ ).

3. Compute |x|. Let k = |x| if x 1 > 0 and k = − |x| if x 1 < 0 (this choice of sign mini- mizes the roundoff error).

4. Let u = k+x 1 x 2 x 3 ···x n−i .

5. Compute H = |u| 2 / 2.

6. Compute v = A ′ u/H.

7. Compute g = u T v/(2H).

8. Compute w = v − gu.

9. Compute the transformation A ← A ′ −w T

u−u T w.

10. Set A i,i+1 =A i+1,i = −k.

Accumulated Transformation Matrix

Since we used similarity transformations, the eigenvalues of the tridiagonal matrix are the same as those of the original matrix. However, to determine the eigenvectors

X of original A we must use the transformation

X = PX tridiag

where P is the accumulation of the individual transformations:

P=P 1 P 2 ···P n−2

We build up the accumulated transformation matrix by initializing P to a n × n identity matrix and then applying the transformation

with i = 1, 2, . . . , n − 2. It can be seen that each multiplication affects only the right- most n − i columns of P (since the first row of P 12 contains only zeroes, it can also be omitted in the multiplication). Using the notation

P 12

P 22

we have &

The procedure for carrying out the matrix multiplication in Eq. (b) is r Retrieve u (in our triangularization procedure the u’s are stored in the columns of the lower triangular portion of A).

r Compute H = |u| 2 / 2.

r Compute y = P ′ u/H. r Compute the transformation P ′ ←P ′ − yu T .

This function performs the Householder reduction on the matrix A. Upon return,

d occupies the principal diagonal of A and c forms the upper subdiagonal; that is,

d = diag(A) and c = diag(A,1) . The portion of A below the principal diagonal is d = diag(A) and c = diag(A,1) . The portion of A below the principal diagonal is

function A = householder(A) % Housholder reduction of A to tridiagonal form A = [c\d\c]. % Extract c and d by d = diag(A), c = diag(A,1). % USAGE: A = householder(A)

n = size(A,1); for k = 1:n-2

u = A(k+1:n,k); uMag = sqrt(dot(u,u)); if u(1) < 0; uMag = -uMag; end u(1) = u(1) + uMag; A(k+1:n,k) = u;

% Save u in lower part of A. H = dot(u,u)/2; v = A(k+1:n,k+1:n)*u/H; g = dot(u,v)/(2*H); v = v - g*u; A(k+1:n,k+1:n) = A(k+1:n,k+1:n) - v*u’ - u*v’; A(k,k+1) = -uMag;

end

The function householderP returns the accumulated transformation matrix P. There is no need to call it if only the eigenvalues are to be computed. Note that the input

parameter A is not the original matrix, but the matrix returned by householder .

function P = householderP(A) % Computes transformation matrix P after % householder reduction of A is carried out. % USAGE: P = householderP(A).

n = size(A,1); P = eye(n); for k = 1:n-2

u = A(k+1:n,k); H = dot(u,u)/2; v = P(1:n,k+1:n)*u/H;

P(1:n,k+1:n) = P(1:n,k+1:n) - v*u’; end

EXAMPLE 9.7 Transform the matrix

9 7 into tridiagonal form using Householder reduction.

Solution Reduce the first row and column:

A ′ ⎢ = ⎣ 5 12 9 ⎥

x= ⎣ 3 ⎥ ⎦

k = |x| = 3. 7417

Q=I− ⎢

10.642 −0.1388 −9.1294 ⎥ ⎥ A← Qx QA ′ Q = ⎢

A 11 (Qx) T ' ⎢ ⎢ −3.7417

In the last step we used the formula Qx = −k 0 · · · 0 .

Reduce the second row and column: &

k = − |x| = −9.1305 4.8429 10.4480

where the negative sign on k was determined by the sign of x 1 .

H= |u| 2 2 = 84.633

85.920 ' 84.623

uu T = 84.623 83.346

uu ' T & 0.01521 −0.99988

QA ′ Q= 4.772 5.762

A 11 A 12 0 T

A← ⎣ A 21 A 22 (Qx) T ⎥ ⎢ −3.742

Use the function householder to tridiagonalize the matrix in Example 9.7; also de- termine the transformation matrix P.

Solution

% Example 9.8 (Householder reduction) A = [7

A = householder(A); d = diag(A)’ c = diag(A,1)’

P = householderP(A)

The results of running the above program are: >> d =