A Modified Newton Method

6.5.5 A Modified Newton Method

There are three competing methods in current use for determining all zeros of a given polynomial. The Jenkins–Traub method [211], used in the IMSL library, is equivalent to a so-called variable-shift Rayleigh quotient iteration for finding the eigenvalues and eigenvectors of the companion matrix. By taking advantage of the matrix structure the work per iteration can be reduced to O(n). A three stage procedure is used, each stage being characterized by the type of shift used. The code CPOLY (see [212]) is available via Netlib; see Sec. C.7 in Online Appendix C. For a description of the Jenkins–Traub method we refer to Ralston and Rabinowitz [296, Sec. 8.11].

The MATLAB zero-finding code roots applies the QR algorithm, which is a standard method for solving eigenvalue problems, to a balanced companion matrix. The balancing involves a diagonal similarity transformation,

A ˜ = DAD −1 , D = diag (d 1 ,d 2 ,...,d n ), which preserves the eigenvalues. The aim is to reduce the norm of A and thereby reduce

the condition number of its eigenvalue problem. The balancing algorithm used is that of Parlett and Reinsch [286].

Another excellent algorithm is the modified Newton algorithm PA16, due to Madsen and Reid [253, 254], used by NAG Library. In its first stage it uses the Newton formula to find a search direction for minimizing |p(z)|. Once the iterates are close to a zero it enters stage 2 and switches to standard Newton. This will be described in more detail below.

Theoretical and experimental comparisons of the three algorithms above are given in [353]. It is shown that the Jenkins–Traub, Madsen–Reid, and QR algorithms all have roughly the same stability properties. The highest accuracy is typically achieved by PA16 and the next best by roots . A possible drawback with the QR algorithm is that it requires

O(n 3 ) work and O(n 2 ) memory. Both the Madsen–Reid and Jenkins–Traub require only O(n 2 ) work O(n) memory. Since one is rarely interested in solving polynomial equations of high degree this is usually not important. We now describe the modified Newton method due to Madsen [253]. By including

a one-dimensional search along the Newton direction this method achieves good global convergence properties and is also effective for multiple roots. To initialize let z 0 = 0,

δz −p(0)/p ′ ( 0) = −a n /a n −1 if a 0 n

1 otherwise, and take

1≤k≤n

|a n |

676 Chapter 6. Solving Scalar Nonlinear Equations This assures that |z 1 | is less than the modulus of any zero of p(z) (see [204, Exercise 2.2.11]).

Further, if p ′ ( Sec. 6.3.2). This choice makes it likely that convergence will take place to a root of near minimal modulus.

The general idea of the algorithm is that given z k , a tentative step h k is computed by Newton’s method. The next iterate is found by taking the best point (in terms of minimizing |f (z)|) found by a short search along the line through z k and z k +h k . When the search yields no better value than at z k we take z k +1 =z k and make sure that the next search is shorter and in a different direction. Since the line searches will be wasteful if we are near a simple root, we then switch to the standard Newton’s method.

In the first stage of the algorithm, when searches are being performed, new iterates z k +1 are computed as follows.

1. If the last iteration was successful (z k k −1 ), then the Newton correction

(6.5.26) is computed. The next tentative step is taken as

h k = −p(z k )/p ′ (z k )

−1 iθ |e h k / |h k | otherwise,

where θ is chosen rather arbitrarily as arctan(3/4). This change of direction is included because if a saddle point is being approached, the direction h k may be a bad choice.

2. If the last step was unsuccessful (z k =z k −1 ) the search direction is changed and the step size reduced. In this case the tentative step is chosen to be

1 iθ δz k =− e δz 2 k −1 .

Repeated use of this is sure to yield a good search direction.

3. Once the tentative step δz k has been found the inequality |p(z k + δz k ) | < |p(z k ) | is tested. If this is satisfied the numbers

|p(z k + p δz k ) |, p = 1, 2, . . . , n, are calculated as long as these are strictly decreasing. Note that if we are close to

a multiple root of multiplicity m we will find the estimate z k + mh k , which gives quadratic convergence to this root. A similar situation will hold if we are at a fair distance from a cluster of m zeros and other zeros are further away.

If |p(z k + δz k ) | ≥ |p(z k ) |, we calculate the numbers

|p(z k +2 −p δz k ) |, p = 0, 1, 2,

again continuing until the sequence ceases to decrease.

6.5. Algebraic Equations 677

A switch to standard Newton is made if in the previous iteration a standard Newton step z k +1 =z k +h k was taken, and Theorem 6.3.3 ensures the convergence of Newton’s method with initial value z k +1 , i.e., when f (z k )f ′ (z k )

2 |f (z k ) | max |f ′′ (z) | ≤ |f ′ (z k ) 2 z ∈K k | , K k : |z − z k | ≤ |h k |, is satisfied; cf. (6.3.20). This inequality can be approximated using already computed

quantities by

2 |f (z k ) ||f ′ (z k ) −f ′ (z k −1 ) | ≤ |f ′ (z k ) 2 | |z k −1 −z k |. (6.5.27) The iterations are terminated and z k +1 accepted as a root whenever z k +1

k and

|z k +1 −z k | < u|z k |

holds, where u is the unit roundoff. The iterations are also terminated if

|p(z k +1 ) | = |p(z k ) | < 16nu|a n |,

where the right-hand side is a generous overestimate of the final roundoff made in computing p(z) at the root of the smallest magnitude. The polynomial is then deflated as described in the previous section.

More details about this algorithm and methods for computing error bounds can be found in [253, 254].