The qd Algorithm

3.5.5 The qd Algorithm

Let {c n } be a sequence of real or complex numbers and

(3.5.46) the formal power series formed with these coefficients. The qd algorithm of Rutishauser

C(z)

=c 2

0 +c 1 z +c 2 z +···

[310] 123 forms from this sequence a two-dimensional array, similar to a difference scheme, by alternately taking difference and quotients as follows. Take as initial conditions

and form the quotient-difference scheme, or qd scheme:

123 Heinz Rutishauser (1912–1970) was a Swiss mathematician, a pioneer in computing, and the originator of many important algorithms. The qd algorithm has had great impact in eigenvalue calculations.

340 Chapter 3. Series, Operators, and Continued Fractions where the quantities are connected by the two rhombus rules

m +1 = (n) q m , m = 1, 2, . . . , n = 0, 1, . . . . (3.5.49)

(n)

(n +1)

Each of the rules connects four adjacent elements of the qd scheme. The first rule states that in any rhombus-like configuration of four elements centered in a q-column the sum of the two NE and the two SW elements are equal. Similarly, the second rule states that in any rhombus-like configuration in an e-column the product of the two NE and the two SW elements are equal.

The initial conditions (3.5.47) give the first two columns in the qd scheme. The remaining elements in the qd scheme, if it exists, can then be generated column by column using the rhombus rules. Note the computations break down if one of the denominators in (3.5.49) is zero. If one of the coefficients c n is zero even the very first q-column fails to exist.

The rhombus rules are based on certain identities between Hankel determinants, which we now describe. These also give conditions for the existence of the qd scheme. The Hankel determinants associated with the formal power series (3.5.46) are, for arbitrary integers n and k ≥ 0, defined by H (n)

··· c n +k−1 ( ( (

( c n +k−1 c n +k−2 ···c n +2k−2 (

where c k = 0 for k < 0. This definition is valid also for negative values of n. Note, however, that if n + k ≤ 0, then the entire first row of H (n)

is zero, i.e.,

A formal power series is called normal if its associated Hankel determinants H (n)

for all m, n ≥ 0; it is called k-normal if H m In the following theorem (Henrici [196, Theorem 7.6a]) the elements in the qd scheme m

(n)

can be expressed in terms of Hankel determinants.

Theorem 3.5.8.

Let H (n)

be the Hankel determinants associated with a formal power series C =

c 0 +c 1 z +c 2 z 2 + · · ·. If there exists a positive integer k such that the series is k-normal, the columns q (n) m of the qd scheme associated with C exist for m = 1 : k, and

H m H m +1) for m = 1 : k and all n ≥ 0. The above result is related to Jacobi’s identity for Hankel matrices, that for all integers

+H k +1 H k −1 = 0. (3.5.53) This identity can be derived from the following very useful determinant identity.

−H (n

(H (n) ) 2 (n −1)

H (n +1)

(n −1)

3.5. Continued Fractions and Padé Approximants 341 Theorem 3.5.9 (Sylvester’s Determinant Identity).

Let ˆ A ∈C n ×n , n ≥ 2, be partitioned: α

Then we have the identity det(A) · det( ˆ A) = det(A 11 ) · det(A 22 ) − det(A 21 ) · det(A 12 ).

Proof. If the matrix A is square and nonsingular, then det(A A

= ± det(A) · (α ij

−a

i A −1 ˆa j ), (3.5.55)

ij

 A  ˆa 11 ˆa 2

det(A) · det( ˆ A) = det(A) · det  a T 1 α 11 α 12 

a T 2 α 21 α 22

2 α 11 α 12 = (det(A)) a · det

a T A −1 ( ˆa 1 ˆa 2 21 ) 22 2 β 11 β = det 12

where β ij =α ij −a T

i A −1 ˆa j . Using (3.5.55) gives (3.5.54), which holds even when A is singular.

If the Hankel determinants H (n) k are arranged in a triangular array,

then Jacobi’s identity links together the entries in a star-like configuration. Since the two first columns are trivial, (3.5.53) may be used to calculate the Hankel determinants recur- sively from left to right. Further properties of Hankel determinants are given in Henrici [196, Sec. 7.5].

342 Chapter 3. Series, Operators, and Continued Fractions We state without proof an important analytical property of the Hankel determinants

that shows how the poles of a meromorphic 124 function can be determined from the coeffi- cients of its Taylor expansion at z = 0.

Theorem 3.5.10.

0 +c 1 z +c 2 z Let f (z) = c 2 + · · · be the Taylor series of a function meromorphic in the disk D : |z| < σ and let the poles z i =u −1 i of f in D be numbered such that

0 < |z 1 | ≤ |z 2 | ≤ · · · < σ.

Then for each m such that |z m | < |z m

+1 (n) |, if n is sufficiently large, H

n lim H (n +1) m (n) /H m =u 1 u 2 ···u m .

In the special case that f is a rational function with a pole of order p at infinity and the sum of orders of all its finite poles is k, then

(3.5.57) where C k

H (n)

=C k (u 1 u 2 ···u k ) n , n > p,

m (n) = 0, n > p, m > k.

Proof. The result is a corollary of Theorem 7.5b in Henrici [196]. The above results are related to the qd scheme as follows; see Henrici [196, Theorem

7.6b].

Theorem 3.5.11.

3.5.10 and assuming that the Taylor series at z = 0 is ultimately k-normal for some integer k >

Under the hypothesis of Theorem

0, the qd scheme for f has the following properties:

(a) For each m such that 0 < m ≤ k and |z m −1 | < |z m | < |z m +1 |,

(b) For each m such that 0 < m ≤ k and |z m | < |z m +1 |,

n lim e →∞ (n) m = 0.

From the above results it seems that, under certain restrictions, an algorithm for simultaneously computing all the poles of a meromorphic function f directly from its Taylor series at the origin could be constructed, where the qd scheme is computed from left to right. Any q-column corresponding to a simple pole of isolated modulus would tend to the reciprocal value of that pole. The e-columns on both sides would tend to zero. If f is rational, the last e-column would be zero, which could serve as a test of accuracy.

A function which is analytic in a region <, except for poles, is said to be meromorphic in <.

3.5. Continued Fractions and Padé Approximants 343 Unfortunately, as outlined, this algorithm is unstable, i.e., oversensitive to rounding

errors, and useless numerically. This fact is related to the occurrence in (3.5.49) of a division of two small quantities, which can have large relative errors. (Recall that e-columns tends to zero.)

A more stable way of constructing the qd scheme is obtained by writing the rhombus rules as

Written in this form, the rules can be used to construct the qd scheme row by row. The problem now is how to start the algorithm. As seen from the scheme below, to do this it suffices to know the first two rows of q’s and e’s. This, together with the first column of zeros, allows us to proceed along diagonals slanted SW; see scheme below.

This is called the progressive form of the qd algorithm. The starting values q (n)

and e m for negative values of n can be computed from the relations (3.5.52). In this form the qd algorithm can be used to simultaneously determine the zeros of a polynomial; see Sec. 6.5.4.

(n) m

The qd algorithm is related to Padé approximants. Consider a continued fraction of the form

c(z) =

The nth approximant

(3.5.61) is the finite continued fraction obtained by setting a n +1 = 0. In the special case that all

w n (z) =P n (z)/Q n (z), n = 1, 2, . . . ,

0, the continued fraction is called a Stieltjes fraction. 125 The sequence of numerators {P n (z) } and denominators {Q n (z) } in (3.5.61) satisfy the recurrence relations

125 The theory of such fractions was first expounded by Stieltjes in a famous memoir which appeared in 1894, the year of his death.

344 Chapter 3. Series, Operators, and Continued Fractions Hence both P n and Q n are polynomials in z of degree ⌊(n − 1)/2⌋ and ⌊n/2⌋, respectively.

It can be shown that the polynomials P n and Q n have no common zero for n = 1, 2, . . . . From the initial conditions and recurrence relations it follows that Q n ( 0) = 1, n =

0, 1, 2, . . . . Hence the rational function w n (z) =P n (z)/Q n (z) is analytic at z = 0. Hence it can be expanded in a Taylor series

0 +c 1 z +c 2 z +···

Q n (z)

that converges for z sufficiently small. The coefficients c (n)

in (3.5.62) can be shown to

(n k

the ultimate value of c k for increasing values n and let

be independent of n for k < n. We denote by c k

:= c (n)

C(z) =c 0 +c 1 z +c 2 z 2 +···

be the formal power series formed with these coefficients. Then the power series C(z) and the fraction c(z) are said to correspond to each other. Note that the formal power series C(z)

The qd algorithm can be used to solve the following problem: Given a (formal) power series C(z) = c 0 +c 1 z +c 2 z 2 + · · ·, find a continued fraction c(z) of the form (3.5.60) corresponding to it. Note that we do not require that the formal power series corresponding

to the continued fraction converges, merely that the nth approximant w n of the continued fraction satisfies

C(z) −w n (z)

= O(z n ).

Theorem 3.5.12 (Henrici [197, Theorem 12.4c]). Given a formal power series C(z) = c 2 0 +c 1 z +c 2 z + · · ·, there exists at most one

corresponding continued fraction of the form

a 0 a 1 z a 2 z a 3 z 4z 1− ···. 1− 1− 1− 1−

There exists precisely one such fraction if and only if the Hankel determinants satisfy H k (n)

and e k are the elements of the qd scheme associated with C, then

Conversely, this shows that knowing the coefficients of the continued fraction cor- responding to f allows us to compute the qd scheme starting from the first diagonal and proceeding in the SW direction. This is called the progressive qd algorithm.

Example 3.5.7.

For the power series

c(z)

2 = 0! + 1!z + 2!z 3 + 3!z +···,

Problems and Computer Exercises 345 we obtain using the rhombus rules (3.5.48)–(3.5.49) the qd scheme

Hence the corresponding continued fraction is

z 2z 2z 3z 3z

c(z) = 1+ ···. 1+ 1+ 1+ 1+ 1+ 1+

ReviewQuestions

3.5.1 Define a continued fraction. Show how the convergents can be evaluated either backward or forward.

3.5.2 Show how any positive number can be expanded into a continued fraction with integer elements. In what sense are the convergents the best approximations? How accurate are they?

3.5.3 What is the Padé table? Describe how the Padé approximants can be computed, if they exist. Tell something about singular and almost singular situations that can be encountered, and how to avoid them.

3.5.4 Describe the ǫ algorithm, and tell something about its background.

3.5.5 What are the rhombus rules for the qd algorithm? What is the difference between the standard and the progressive qd algorithm?

3.5.6 Sketch how the qd algorithm, under some restrictions, can be used to compute the zeros of a polynomial. Give necessary conditions for this to work. What governs the rate of convergence?

Problems and Computer Exercises

3.5.1 (a) Write a program for the algorithm of best rational approximations to a real number in Sec. 3.5.1.

346Chapter 3. Series, Operators, and Continued Fractions Apply it to find a few coefficients of the continued fractions for

2, e, π, log 2 , 2 j/ 2 12 5 + 1), log 3

for a few integers j, 1 ≤ j ≤ 11. (b) Check the accuracy of the convergents. What happens when you apply your program to a rational number, e.g., 729/768?

(c) The metonic cycle used for calendrical purposes by the Greeks consists of 235 lunar months, which nearly equal 19 solar years. Show, using the algorithm in Sec. 3.5.1, that 235/19 is the sixth convergent of the ratio 365.2495/29.53059 of solar period and the lunar phase (synodic) period.

3.5.2 A matrix formalism for continued fractions. (a) We use the same notations as in Sec. 3.5.1, but set, with no loss of generality,

Show that P (0) = I, P (n) = P (n − 1)A(n),

P (n) = A(1)A(2) · · · A(n − 1)A(n), n ≥ 1. Comment: This does not minimize the number of arithmetic operations but, in a

matrix-oriented programming language, it often gives very simple programs. (b) Write a program for this with some termination criterion and test it on a few

cases, such as

2+ 1+ ···. 1+ 1+ 3+ 2+ 3+ 2+ 3+ 2+ 3+ 4+ As a postprocessing, apply Aitken acceleration in the first two cases in order to

obtain a very high accuracy. Does the result look familiar in the last case? See Problem 3.5.3 concerning the exact results in the two other cases.

(c) Write a version of the program with some strategy for scaling P (n) in order to eliminate the risk of overflow and underflow.

Hint: Note that the convergents x n =p n /q n are unchanged if you multiply the P (n) by arbitrary scalars.

(d) Use this matrix form for working out a short proof of (3.5.7). Hint: What is the determinant of a matrix product?

3.5.3 (a) Explain that x = 1 + 1/x for the continued fraction in (3.5.13). (b) Compute the periodic continued fraction

exactly (by paper and pencil). (The convergence is assured by Seidel’s theorem (Theorem 3.5.2).)

Problems and Computer Exercises 347 (c) Suggest a generalization of (a) and (b), where you can always obtain a quadratic

equation with a positive root. (d) Show that

···. x

where y =

− x −y

3.5.4 (a) Prove the equivalence transformation (3.5.8). Show that the errors of the con- vergents have alternating signs if the elements of the continued fraction are positive. (b) Show how to bring a general continued fraction to the special form of equation

3.5.5 Show that the (1, 1) Padé approximant of

1 + x equals (4 + 3x)/(4 + x). What is the (2, 2) Padé approximant?

3.5.6 Let P m,m (z)/Q m,m (z)

be the diagonal Padé approximants of the exponential func- tion.

(a) Show that the coefficients for P m,m (z) satisfy the recursion

p j , j = 0 : m − 1. (3.5.65)

( 2m − j)(j + 1)

(b) Show that for m = 6 we have

1 5 1 1 4 1 1 P 6,6 (z)

2 3 5 =1+ 6 z + z + z + z + z + z

15,840 665,280 and Q 6,6 (z) =P 6,6 ( −z). How many operations are needed to evaluate this approx-

imation for a given z? (c) Use the error estimate in (3.5.35), neglecting higher-order terms, to compute a bound for the relative error of the approximation in (b) when |z| ∈ [0, ln 2]. What

degree of the diagonal Padé approximant is needed for the relative error to be of the order of the unit roundoff 2 −53 = 1.11 · 10 −16 in IEEE double precision arithmetic?

3.5.7 For k = 1, Shanks’ sequence transformation (3.5.40) becomes

Show that this is mathematically equivalent to the result s n ′ +2 from Aitken extrapo- lation. Why is the direct use of the above expression not safe numerically?

3.5.8 (a) Write a program for computing a Padé approximant and its error term. Apply it (perhaps after a transformation) for various values of m, n to, e.g., e z , arctan z, tan z. (Note that two of these examples are odd functions.) Use the algorithm of Sec. 3.5.1 for expressing the coefficients as rational numbers. For how large m and n can you use your program (in these examples) without severe trouble with rounding errors?

(b) Let m be an odd number. Try to transform the (m, m + 1) Padé approximants of arctan z and tan z to continued fractions of the form given in Sec. 3.5.1.

(c) Try to determine for which other functions the Padé table has a similar symmetry as shown in the text for the exponential function e z .

348 Chapter 3. Series, Operators, and Continued Fractions

3.5.9 (a) Show that there is at most one rational function R(z), where the degrees of the numerator and denominator do not exceed, respectively, m and n such that

− R(z) = O(z m +n+1 ) as z → 0, even if the system (3.5.28) is singular. (Note, however, that P m and Q n are not

f (z)

uniquely determined if the system is singular; they have common factors.) (b) Is it true that if f (z) is a rational function of degrees m ′ ,n ′ , then

[m, n] f (z) = f (z) ∀ m ≥ m ′ , n ≥n ′ ?

3.5.10 Write a program for evaluating the incomplete gamma function. Use the continued fraction (3.5.22) for x greater than about a + 1. For x less than about a + 1 use the

power series for γ (a, x).

3.5.11 1 1 1 Compute the infinite sum 1 − 1 3 + 5 − 7 + 9 − · · · with the epsilon algorithm, and estimate (empirically) the speed of convergence.

3.5.12 Write a program for determining the zeros of a polynomial p(z) of degree n with simple positive zeros. Test it by computing the zeros of some orthogonal polyno- mials. Discuss how you can shift the zeros so that convergence to a particular zero is enhanced.