Jacobi Method

9.2 Jacobi Method

Similarity Transformation and Diagonalization

Consider the standard matrix eigenvalue problem

(9.4) where A is symmetric. Let us now apply the transformation

Ax = λx

(9.5) where P is a nonsingular matrix. Substituting Eq. (9.5) into Eq. (9.4) and premultiplying

x = Px ∗

each side by P −1 , we get

P −1 APx ∗ = λP −1 Px ∗

or

A ∗ x ∗ = λx ∗

Similarity transformations are frequently used to change an eigenvalue problem to a form that is easier to solve. Suppose that we managed by some means to find a P that diagonalizes A ∗ , so that Eqs. (9.6) are

The solution of these equations is λ 1 =A ∗ 11 λ 2 =A ∗ 22 ··· λ n =A ∗ nn

X ∗ = x ∗ 1 x ∗ 2 ···x ∗ n =I

According to Eq. (9.5) the eigenvector matrix of A is

(9.8) Hence the transformation matrix P is the eigenvector matrix of A and the eigenvalues

X = PX ∗ = PI = P

of A are the diagonal terms of A ∗ .

Jacobi Rotation

A special transformation is the plane rotation

x = Rx ∗

(9.11) One consequence of orthogonality is that the transformation in Eq. (9.9) has the

R −1 =R T

essential characteristic of a rotation: it preserves the magnitude of the vector; that is, |x| = |x ∗ |.

The similarity transformation corresponding to the plane rotation in Eq. (9.9) is

(9.12) The matrix A ∗ not only has the same eigenvalues as the original matrix A, but due to

A ∗ =R −1

AR = R T AR

orthogonality of R it is also symmetric. The transformation in Eq. (9.12) changes only the rows/columns k and ℓ of A. The formulas for these changes are

A ∗ kk =c 2 A kk +s 2 A ℓℓ − 2cs A kℓ

A ∗ ℓℓ =c 2 A ℓℓ +s 2 A kk + 2cs A kℓ

A ∗ kℓ =A ∗ ℓ k = (c 2 −s 2 )A kℓ + cs(A kk −A ℓℓ ) (9.13)

A ∗ ki =A ∗ ik =cA ki −sA ℓ i ,

A ∗ ℓ i =A ∗ iℓ =cA ℓ i +sA ki ,

Jacobi Diagonalization

The angle θ in the Jacobi rotation matrix can be chosen so that A ∗ kℓ =A ∗ ℓ k = 0. This sug- gests the following idea: why not diagonalize A by looping through all the off-diagonal terms and eliminate them one by one? This is exactly what Jacobi diagonalization does.

However, there is a major snag—the transformation that annihilates an off-diagonal term also undoes some of the previously created zeroes. Fortunately, it turns out that the off-diagonal terms that reappear will be smaller than before. Thus Jacobi method is an iterative procedure that repeatedly applies Jacobi rotations until the off-diagonal terms have virtually vanished. The final transformation matrix P is the accumulation of individual rotations R i :

(9.14) The columns of P finish up being the eigenvectors of A and the diagonal elements of

P=R 1 ·R 2 ·R 3 ···

A ∗ =P T AP become the eigenvectors. Let us now look at the details of a Jacobi rotation. From Eq. (9.13) we see that

A ∗ kℓ = 0 if

(c 2 −s 2 )A kℓ + cs(A kk −A ℓℓ )=0

(a) Using the trigonometric identities c 2 −s 2 = cos 2θ and cs = (1/2) sin 2θ, we obtain

from Eq. (a)

which could be solved for θ , followed by computation of c = cos θ and s = sin θ. How- ever, the procedure described below leads to a better algorithm. 20

Introducing the notation

and utilizing the trigonometric identity

2t tan 2θ = (1 − t 2 )

where t = tan θ, we can write Eq. (b) as

t 2 + 2φt − 1 = 0

which has the roots

t = −φ ± φ 2 +1

It has been found that the root |t| ≤ 1, which corresponds to |θ| ≤ 45 ◦ , leads to the more stable transformation. Therefore, we choose the plus sign if φ > 0 and the minus

20 The procedure is adapted from W.H. Press et al., Numerical Recipes in Fortran, 2nd ed. (1992), Cambridge University Press.

sign if φ ≤ 0, which is equivalent to using

t = sgn(φ) − |φ| + φ 2 +1

To forestall excessive roundoff error if φ is large, we multiply both sides of the equation by |φ| + φ 2 + 1 and solve for t, which yields

|φ| + φ 2 +1 In the case of very large φ, we should replace Eq. (9.16a) by the approximation

to prevent overflow in the computation of φ 2 . Having computed t, we can use the

trigonometric relationship tan θ = sin θ/ cos θ =

1 − cos 2 θ/ cos θ to obtain

We now improve the computational properties of the transformation formulas in Eqs. (9.13). Solving Eq. (a) for A ℓℓ , we obtain

Replacing all occurrences of A ℓℓ by Eq. (c) and simplifying, we can write the transfor- mation formulas in Eqs. (9.13) as

The introduction of τ allowed us to express each formula in the form (original value) + (change), which is helpful in reducing the roundoff error.

At the start of Jacobi’s diagonalization process the transformation matrix P is initialized to the identity matrix. Each Jacobi rotation changes this matrix from P to P ∗ = PR. The corresponding changes in the elements of P can be shown to be

(only the columns k and ℓ are affected)

We still have to decide the order in which the off-diagonal elements of A are to be eliminated. Jacobi’s original idea was to attack the largest element since this results in fewest number of rotations. The problem here is that A has to be searched for the largest element after every rotation, which is a time-consuming process. If the matrix is large, it is faster to sweep through it by rows or columns and annihilate every element above some threshold value. In the next sweep the threshold is lowered and the process repeated. We adopt Jacobi’s original scheme because of its simpler implementation.

In summary, the Jacobi diagonalization procedure, which uses only the upper half of the matrix, is

1. Find the largest (absolute value) off-diagonal element A kℓ in the upper half of A.

2. Compute φ, t, c and s from Eqs. (9.15)–(9.17).

3. Compute τ from Eq. (9.19).

4. Modify the elements in the upper half of A according to Eqs. (9.18).

5. Update the transformation matrix P using Eqs. (9.20). Repeat the procedure until the A kℓ <ε , where ε is the error tolerance.

The function jacobi computes all eigenvalues λ i and eigenvectors x i of a symmetric, n × n matrix A by the Jacobi method. The algorithm works exclusively with the upper

triangular part of A, which is destroyed in the process. The principal diagonal of A is replaced by the eigenvalues, and the columns of the transformation matrix P become the normalized eigenvectors.

function [eVals,eVecs] = jacobi(A,tol) % Jacobi method for computing eigenvalues and % eigenvectors of a symmetric matrix A. % USAGE: [eVals,eVecs] = jacobi(A,tol) % tol = error tolerance (default is 1.0e-9).

if nargin < 2; tol = 1.0e-9; end n = size(A,1); maxRot = 5*(nˆ2);

% Limit number of rotations

P = eye(n); % Initialize rotation matrix for i = 1:maxRot

% Begin Jacobi rotations [Amax,k,L] = maxElem(A); if Amax < tol;

eVals = diag(A); eVecs = P; return

end [A,P] = rotate(A,P,k,L);

end error(’Too many Jacobi rotations’)

function [Amax,k,L] = maxElem(A) % Finds Amax = A(k,L) (largest off-diag. elem. of A). n = size(A,1); Amax = 0; for i = 1:n-1

for j = i+1:n if abs(A(i,j)) >= Amax Amax = abs(A(i,j)); k = i; L = j;

end end end

function [A,P] = rotate(A,P,k,L) % zeros A(k,L) by a Jacobi rotation and updates % transformation matrix P. n = size(A,1); diff = A(L,L) - A(k,k); if abs(A(k,L)) < abs(diff)*1.0e-36

t = A(k,L); else phi = diff/(2*A(k,L)); t = 1/(abs(phi) + sqrt(phiˆ2 + 1)); if phi < 0; t = -t; end;

end c = 1/sqrt(tˆ2 + 1); s = t*c; tau = s/(1 + c); temp = A(k,L); A(k,L) = 0; A(k,k) = A(k,k) - t*temp;

A(L,L) = A(L,L) + t*temp; for i = 1:k-1

% For i < k

temp = A(i,k); A(i,k) = temp -s*(A(i,L) + tau*temp); A(i,L) = A(i,L) + s*(temp - tau*A(i,L));

end for i = k+1:L-1

% For k < i < L

temp = A(k,i); A(k,i) = temp - s*(A(i,L) + tau*A(k,i)); A(i,L) = A(i,L) + s*(temp - tau*A(i,L));

end for i = L+1:n

% For i > L

temp = A(k,i); A(k,i) = temp - s*(A(L,i) + tau*temp); A(L,i) = A(L,i) + s*(temp - tau*A(L,i));

end for i = 1:n

% Update transformation matrix

temp = P(i,k); P(i,k) = temp - s*(P(i,L) + tau*P(i,k)); P(i,L) = P(i,L) + s*(temp - tau*P(i,L));

end

The eigenvalues/eigenvectors returned by jacobi are not ordered. The function listed below can be used to sort the results into ascending order of eigenvalues.

function [eVals,eVecs] = sortEigen(eVals,eVecs) % Sorts eigenvalues & eigenvectors into ascending % order of eigenvalues. % USAGE: [eVals,eVecs] = sortEigen(eVals,eVecs)

n = length(eVals); for i = 1:n-1

index = i; val = eVals(i); for j = i+1:n

if eVals(j) < val index = j; val = eVals(j); end end if eVals(j) < val index = j; val = eVals(j); end end

end end

Transformation to Standard Form

Physical problems often give rise to eigenvalue problems of the form

(9.21) where A and B are symmetric n × nmatrices. We assume that B is also positive definite.

Ax = λBx

Such problems must be transformed into the standard form before they can be solved by Jacobi diagonalization.

As B is symmetric and positive definite, we can apply Choleski’s decomposition

B = LL T , where L is a lower-triangular matrix (see Art. 2.3). Then we introduce the transformation

(9.22) Substituting into Eq. (9.21), we get

−1 ) x = (L T z

A(L −1 ) T

z =λLL T (L −1 ) T z

Premultiplying both sides by L −1 results in

L −1 A(L −1 ) T

z = λL −1 LL T (L −1 ) T z

Because L −1 L=L T (L −1 ) T = I, the last equation reduces to the standard form

(9.23) where

Hz = λz

(9.24) An important property of this transformation is that it does not destroy the symmetry

−1 H=L T A(L )

of the matrix; i.e., a symmetric A results in a symmetric H. Here is the general procedure for solving eigenvalue problems of the form Ax = λBx:

1. Use Choleski’s decomposition B = LL T to compute L.

2. Compute L −1 (a triangular matrix can be inverted with relatively small computa- tional effort).

3. Compute H from Eq. (9.24).

4. Solve the standard eigenvalue problem Hz = λz (e.g., using the Jacobi method).

5. Recover the eigenvectors of the original problem from Eq. (9.22): x = (L −1 ) T z. Note that the eigenvalues were untouched by the transformation.

An important special case is where B is a diagonal matrix:

···β n −1/2 0 0 n and

H ij =A ij i β −1/2 j

(9.26b)

Given the matrices A and B, the function stdForm returns H and the transformation −1 ) matrix T = (L T . The inversion of L is carried out by the subfunction invert (the

triangular shape of L allows this to be done by back substitution).

function [H,T] = stdForm(A,B) % Transforms A*x = lambda*B*x to H*z = lambda*z % and computes transformation matrix T in x = T*z. % USAGE: [H,T] = stdForm(A,B)

n = size(A,1); L = choleski(B); Linv = invert(L);

H = Linv*(A*Linv’); T = Linv’;

function Linv = invert(L) % Inverts lower triangular matrix L. n = size(L,1); for j = 1:n-1

L(j,j) = 1/L(j,j); L(j,j) = 1/L(j,j);

L(i,j) = -dot(L(i,j:i-1), L(j:i-1,j)/L(i,i));

end end L(n,n) = 1/L(n,n); Linv = L;

The stress matrix (tensor) corresponding to the state of stress shown is

(each row of the matrix consists of the three stress components acting on a coordinate plane). It can be shown that the eigenvalues of S are the principal stresses and the eigenvectors are normal to the principal planes. (1) Determine the principal stresses by diagonalizing S with a Jacobi rotation and (2) compute the eigenvectors.

Solution of Part (1) To eliminate S 12 we must apply a rotation in the 1–2 plane. With k = 1 and ℓ = 2 Eq. (9.15) is

Equation (9.16a) then yields

sgn(φ)

2 +1 = −0.53518 According to Eqs. (9.18), the changes in S due to the rotation are S ∗ 11 =S 11 − tS 12 = 80 − (−0.53518) (30) = 96.055 MPa S ∗ 22 =S 22 + tS 12 = 40 + (−0.53518) (30) = 23.945 MPa

t=

S ∗ 12 =S ∗ 21 =0

Hence the diagonalized stress matrix is

where the diagonal terms are the principal stresses. Solution of Part (2) To compute the eigenvectors, we start with Eqs. (9.17) and (9.19),

which yield

s = tc = (−0.53518) (0.88168) = −0.47186

We obtain the changes in the transformation matrix P from Eqs. (9.20). Because P is initialized to the identity matrix (P ii = 1 and P ij

= 0 − (−0.47186) [1 + (−0.25077) (0)] = 0.47186 Similarly, the second equation of Eqs. (9.20) yields

P ∗ = −0.47186

P 12 ∗ 22 = 0.88167

The third row and column of P are not affected by the transformation. Thus

0.88167 ⎤ −0.47186 0 P ∗ ⎢ = ⎣ 0.47186

The columns of P ∗ are the eigenvectors of S. EXAMPLE 9.2

2L

i 1 3C i 2 C i 3 C i 1 i 2 i 3

(1) Show that the analysis of the electric circuit shown leads to a matrix eigenvalue problem. (2) Determine the circular frequencies and the relative amplitudes of the currents.

Solution of Part (1) Kirchoff’s equations for the three loops are

Differentiating and substituting dq k / dt = i k , we get

d 2 i 3 −i 2 + 2i 3 = −2LC dt 2

These equations admit the solution

i k (t) = u k sin ωt

where ω is the circular frequency of oscillation (measured in rad/s) and u k are the relative amplitudes of the currents. Substitution into Kirchoff’s equations yields Au = λBu (sin ωt cancels out), where

which represents an eigenvalue problem of the nonstandard form. Solution of Part (2) Since B is a diagonal matrix, we can readily transform the problem

into the standard form Hz = λz. From Eq. (9.26a) we get

L −1 = ⎢ ⎣ 0 1 0 ⎥ √ ⎦

and Eq. (9.26b) yields

H= ⎣ −1/3

The eigenvalues and eigenvectors of H can now be obtained with the Jacobi method. Skipping the details, we obtain the following results:

0.59444 The eigenvectors of the original problem are recovered from Eq. (9.22): y i = (L −1 ) T z i ,

which yields ⎡

0.42033 These vectors should now be normalized (each z i was normalized, but the transfor-

mation to u i does not preserve the magnitudes of vectors). The circular frequencies are ω

i = λ i / (LC ), so that

LC EXAMPLE 9.3

The propped cantilever beam carries a compressive axial load P. The lateral displace- ment u(x) of the beam can be shown to satisfy the differential equation

(a) where E I is the bending rigidity. The boundary conditions are

u(0) = u ′ (0) = 0 u(L) = u (L) = 0 (b) (1) Show that displacement analysis of the beam results in a matrix eigenvalue

problem if the derivatives are approximated by finite differences. (2) Use the Jacobi method to compute the lowest three buckling loads and the corresponding eigenvectors.

Solution of Part (1) We divide the beam into n + 1 segments of length L/(n + 1) each as shown and enforce the differential equation at nodes 1 to n. Replacing the

derivatives of u in Eq. (a) by central finite differences of O(h 2 ) at the interior nodes

(nodes 1 to n), we obtain

After multiplication by h 4 , the equations become

u −1 − 4u 0 + 6u 1 − 4u 2 +u 3 = λ(−u 0 + 2u 1 −u 2 ) u 0 − 4u 1 + 6u 2 − 4u 3 +u 4 = λ(−u 1 + 2u 2 −u 3 )

(c) u n−3 − 4u n−2 + 6u n−1 − 4u n +u n+1 = λ(−u n−2 + 2u n−1 −u n ) u n−2 − 4u n−1 + 6u n − 4u n+1 +u n+2 = λ(−u n−1 + 2u n −u n+1 )

The displacements u −1 ,u 0 ,u n+1 and u n+2 can be eliminated by using the prescribed boundary conditions. Referring to Table 8.1, we obtain the finite difference approxi- mations to the boundary conditions in Eqs. (b):

u n+2 =u n Substitution into Eqs. (c) yields the matrix eigenvalue problem Ax = λBx, where

u 0 =0

u −1 = −u 1 u n+1 =0

0 ⎥ A= ⎥

B= ⎥

Solution of Part (2) The problem with the Jacobi method is that it insists on finding all the eigenvalues and eigenvectors. It is also incapable of exploiting banded structures of matrices. Thus the program listed below does much more work than necessary for the problem at hand. More efficient methods of solution will be introduced later in this chapter.

% Example 9.3 (Jacobi method) n = 10;

% Number of interior nodes. A = zeros(n); B = zeros(n);

% Start constructing A and B. for i = 1:n A(i,i) = 6; B(i,i) = 2; end A(1,1) = 5; A(n,n) = 7; for i = 1:n-1

A(i,i+1) = -4; A(i+1,i) = -4; B(i,i+1) = -1; B(i+1,i) = -1;

end for i = 1:n-2

A(i,i+2) = 1; A(i+2,i) = 1; end [H,T] = stdForm(A,B);

% Convert to std. form. [eVals,Z] = jacobi(H);

% Solve by Jacobi method. X = T*Z;

% Eigenvectors of orig. prob. for i = 1:n

% Normalize eigenvectors. xMag = sqrt(dot(X(:,i),X(:,i))); X(:,i) = X(:,i)/xMag;

end [eVals,X] = sortEigen(eVals,X); % Sort in ascending order. eigenvalues = eVals(1:3)’

% Extract 3 smallest eigenvectors = X(:,1:3)

% eigenvalues & vectors.

Running the program resulted in the following output: >> eigenvalues =

eigenvectors = 0.1641

The first three mode shapes, which represent the relative displacements of the buckled beam, are plotted below (we appended the zero end displacements to the eigenvectors before plotting the points).

The buckling loads are given by P

i = (n + 1) λ i

E I /L 2 . Thus

EI

(11) 2 (0.1641) E I

L 2 = 19.86 L 2

EI

(11) 2 (0.4720) E I

L 2 = 57.11 L 2

EI

(11) 2 (0.9022) E I

L 2 = 109.2 L 2

The analytical values are P 1 = 20.19E I/L 2 ,P 2 = 59.68E I/L 2 and P 3 = 118.9E I/L 2 . It can be seen that the error introduced by the finite element approximation increases with the mode number (the error in P i+1 is larger than in P i ). Of course, the accuracy of the finite difference model can be improved by using larger n, but beyond n = 20 the cost of computation with the Jacobi method becomes rather high.