Fast Algorithms for Vandermonde Systems

4.2.5 Fast Algorithms for Vandermonde Systems

Given distinct scalars x 1 ,x 2 ,...,x n , the coefficients for the interpolating polynomial in the

power basis p = i

=1 a i x are given by the solution to the linear system

(4.2.36) where V is the Vandermonde matrix

V = V (x 1 ,x 2 ,...,x n ) = 

1 n 2 ···x

n −1

Vandermonde systems are appropriate to use for polynomial interpolation when the result is to be expressed in the monomial basis. Often the Newton basis is a better choice. As we shall see, even for solving the Vandermonde system (4.2.36) an algorithm based on Newton’s interpolation formula is a much better choice than Gaussian elimination.

Newton’s interpolation formula gives the interpolation polynomial in the form

p(x) =c 1 p 1 (x) +c 2 p 2 (x) +···+c n p n (x), where the basis polynomials are p 1 (x) = 1,

p k (x) = (x − x 1 ) · · · (x − x k −1 ), k = 2 : n. Here c j = [x 1 ,...,x j −1 ]f and the divided differences can be recursively computed as

described in Sec. 4.2.1. Then the coefficient vector a of p(x) in the power basis

p(x)

=a n 1 +a 2 x +···+a n x −1

can be computed by Horner’s rule. Note that the matrix V T is never formed and only storage for a few vectors is needed. It is easily verified that the operation count is 5 2 n(n + 1) flops. The related primal Vandermonde system

(4.2.38) arises in related problems of determining approximation of linear functionals.

Vy =b

374 Chapter 4. Interpolation and Approximation

Example 4.2.6.

0 x −1/2 f (x) dx that is exact for f ∈ P n and uses values f (x i ) , i = 1 : n. Such integrals need a special treatment

We shall find a formula for integrals of the form I (f ) = 1

because of the singularity at x = 0. Set µ j = x −1/2

0 x −1 dx and introduce the row vector µ T

= (µ n 1 ,µ 2 ,...,µ n )

. We have that f (x) ≈ p(x) = T

i =1 c j x −1 , where V c = f , and

I (f ) ≈

x −1/2 p(x) dx =

=µ T V n −T f, (4.2.39)

0 i =1

where V n is the Vandermonde basis. This can be written I (f ) = a T f , where the coefficient vector is a = V n −1 µ , i.e., a is the solution to the primal Vandermonde system V a = µ.

Clearly the approach in Example 4.2.6 can be used for any linear functional. We would also like to have a stable and efficient method for solving the primal system. One possibility would be to use the algorithm given in Sec. 4.2.3 which computes the inverse

V −1 in about 6n 2 operations, and then form the product V −1 b = y. We shall now derive a more efficient and accurate algorithm for solving primal Vander- monde systems. We start by expressing the solution of the dual problem in terms of a matrix factorization. Using the power basis the unique polynomial satisfying the interpolation conditions p(x i ) =f i , i = 1 : n, is

p(x)

= (1, x, . . . , x n −1 )a,

where the coefficient vector a satisfies the linear system V T a =f. To derive a corresponding algorithm for solving primal Vandermonde systems the above algorithm can be interpreted as a factorization of the matrix (V T ) −1 into a product of diagonal and lower bidiagonal matrices. Let

D k = diag (1, . . . , 1, (x k +1 −x 1 ), . . . , (x n −x n −k )) (4.2.40) and define the matrices

L k (x)

= k −1

, k = 1 : n − 1, (4.2.41)

0 B n −k+1 (x)

Then the dual Vandermonde algorithm can be written in matrix terms as c = U T f ,a=L c , where

1 1 2 (x 2 ) ···L n −1 (x n −1 ).

Since a = V −T f =L T U T f , we have V −T

=L T U T .

4.2. Interpolation Formulas and Algorithms 375 Algorithm 4.1. Fast Dual Vandermonde Solver.

function a = dvand(x,f); n = length(x); a = f; % Compute divided differences of f(1:n). for k = 1:n-1

for j = n:(-1):k+1

a(j) = (a(j) - a(j-1))/(x(j) - x(j-k)); end end % Compute coefficients using Horner’s scheme. for k = n-1:(-1):1

for j = k:n-1 a(j) = a(j) - x(k)*a(j+1); end end

We can now obtain a fast algorithm for solving a primal Vandermonde system V y = b as follows. Transposing the matrix factorization of V −T gives V −1 = UL. Hence y =

V −1 b = U(Lb) and the solution to the primal system can be computed from d = Lb, y = Ud. Transposing (4.2.43)–(4.2.44) gives

L =L n −1 (x n −1 ) ···L 2 (x 2 )L 1 (x 1 ),

=L T 1 ( 1)D −1

1 ···L n −1 ( 1)D n −1 −1 .

This leads to an algorithm for solving primal Vandermonde systems. The operation count and storage requirement of this are the same as for Algorithm 4.1.

Algorithm 4.2. Fast Primal Vandermonde Solver. function y = pvand(x,b);

n = length(x); y = b; for k = 1:n-1

for j = n:(-1):k+1 y(j) = y(j) - x(k)*y(j-1); end end for k = n-1:(-1):1

for j = k+1:n y(j) = y(j)/(x(j) - x(j-k)); end for j = k:n -1 y(j) = y(j) - y(j+1); end end

376Chapter 4. Interpolation and Approximation The given algorithms can be generalized to confluent Vandermonde matrices (see

Example 4.3.1). The above two algorithms are not only fast. They also give almost full relative accuracy in the solution of some Vandermonde systems which are so ill-conditioned that Gaussian elimination with complete pivoting fails to produce a single correct digit. This was first observed by Björck and Pereyra [31], from which the following example is taken.

Example 4.2.7.

Consider a primal Vandermonde system V n y = b, with x i = 1/(i + 2),

= 1/2 i −1 , i = 1 : n. The exact solution can be shown to be

i = (−1) −1 n (

1 + i/2) −1 .

Let ¯y i

be the solution computed by the primal Vandermonde algorithm and take as a measure of the relative error e n = max 1≤i≤n |y i − ¯y i |/|y i |. Using a hexadecimal floating-point arithmetic with u = 16 −13 = 2.22 · 10 −16 , the following results were obtained.

e n /u 4 5 10 54 81

The computed solution has small componentwise relative error which is remarkable since, for example, κ(V 10 )

A forward error analysis given by Higham [198] explains the surprisingly favorable results. If the points are positive and monotonically ordered,

(4.2.45) then the error in the solution ¯a of a Vandermonde system V y = b computed by the primal

0<x 1 <x 2 < ···<x n ,

algorithm can be bounded as

(4.2.46) If the components of the right-hand side satisfy (−1) n b

| ¯a − a| ≤ 5u|V −1

| |b| + O(u 2 ).

i ≥ 0, then |V −1 | |b| = |V −1 b |, and this bound reduces to

(4.2.47) i.e., the solution is computed with small relative error independent of the conditioning of

| ¯a − a| ≤ 5u|a| + O(u 2 ) ;

V . A similar result holds for the dual algorithm. These good results can be shown to be related to the fact that when (4.2.45) holds, the matrix V (x 1 ,x 2 ,...,x n ) is totally positive, i.e., the determinant of every square’s submatrix of V is positive; see [135, 42, 95]. Fast Björck–Pereyra-type algorithms for Vandermonde-like matrices of the form

P = (p i (α j )) n i,j =1 ,

4.2. Interpolation Formulas and Algorithms 377 where p i (x) , i = 1 : n, are basis polynomials in P n that satisfy a three-term recurrence

relation, have also been developed; see Higham [199, Sec. 22.2]. Cauchy linear systems Cx = b, where the elements of C have the special form

c ij = 1/(x i +y j ),

1 ≤ i, j ≤ n,

can also be solved with an O(n 2 ) Björck–Pereyra-type algorithm; see Boros, Kailath, and Olshevsky [42].