Newton’s Interpolation Formula

4.2.1 Newton’s Interpolation Formula

Newton’s interpolation formula uses the triangle family of Newton polynomials (4.1.10). Let p ∈ P n

be the unique polynomial interpolating a given function f (x) at n distinct real

4.2. Interpolation Formulas and Algorithms 359 or complex points x 1 ,x 2 ,...,x n . Suppose that an expansion

f (x) =c 1 +c 2 (x −x 1 ) +···+c n (x −x 1 )(x −x 2 ) · · · (x − x n −1 )

(4.2.1) holds, where A n (x) is the coefficient of the remainder term. If f ∈ P n , we know from

+A n (x)(x −x 1 )(x −x 2 ) · · · (x − x n )

Sec. 4.1.2 that such a formula holds with A n (x) ≡ 0. We shall see that it is correct in general.

For x = x 1 we get c 1 = f (x 1 ) . Further,

[x 1 ,x ]f = c 2 +c 3 (x −x 2 ) +···+c n (x −x 2 ) · · · (x − x n −1 )

+A n (x)(x −x 2 ) · · · (x − x n ),

where we have set

f (x) − f (x 1 )

[x 1 ,x ]f =

x −x 1 This shows that c 2 = [x 1 ,x 2 ]f .

We now define divided differences 128 for k > 1, by the recursion [x 1 ,...,x k −1 ,x ]f − [x 1 ,...,x k

−1 ,x k ]f k −1 x k ,x ]f =

[x 1 ,...,x

x −x k

We obtain, for k = 2, [x 1 ,x 2 ,x ]f = c 3 +c 4 (x −x 3 ) +···+c n (x −x 3 ) · · · (x − x n −1 )

+A n (x)(x −x 3 ) · · · (x − x n ),

and c 3 = [x 1 ,x 2 ,x 3 ]f . By induction it follows that

(4.2.3) i.e., c k in (4.2.1) equals the (k − 1)th divided difference of f . Further,

c k = [x 1 ,...,x k −1 ,x k ]f, k = 1 : n;

A n (x) = [x 1 ,x 2 ,...,x n ,x ]f

for the coefficient of the remainder term. We are now ready to state Newton’s interpolation formula with exact remainder.

Theorem 4.2.1.

The unique interpolation polynomial p ∈ P n such that p(x i ) = f (x i ), i = 1 : n, where the x i are distinct points, can be written as

p(x) =

c k M k −1 (x),

k =1

where c k = [x 1 ,x 2 ,...,x k ]f is the divided difference and M 0 = 1,

M k (x) =M k −1 (x)(x −x k ), k = 1 : n. (4.2.5)

128 We prefer the modern notation [. . .]f to the older notations f [. . .] or f (. . .), since it emphasizes that [. . .] is an operator. Note that the interpretation [x]f = f (x) is consistent with this.

360 Chapter 4. Interpolation and Approximation The formula

f (x) = [x 1 ,x 2 ,...,x k ]f M k −1 (x) + [x 1 ,x 2 ,...,x n ,x ]f M n (x) (4.2.6)

k =1

is an identity, i.e., the exact remainder equals

f (x) − p(x) = [x 1 ,x 2 ,...,x n ,x ]f M n (x). (4.2.7) These formulas are valid also for complex x i and x.

Proof. For f ∈ P n we know that (4.2.1) is correct, hence we can trust the coefficients

c k = −[x 1 ,...,x k ]f . Moreover, since p(x k ) = f (x k ) , k = 1 : n, it follows that

[x 1 ,x 2 ,...,x k ]p = [x 1 ,x 2 ,...,x k ]f , and hence (4.2.4) holds.

We prove the identity (4.2.6) by induction. For n = 1 it is true because the right-hand side becomes f (x 1 ) + [x 1 ,x ]f · (x − x 1 ) = f (x), which equals the left-hand side. Next

suppose that it is true for n = m. The difference between the right-hand side for n = m + 1 and n = m is

[x 1 ,...,x m +1 ]f − [x 1 ,...,x m ,x ]f M m (x) + [x 1 ,...,x m +1 ,x ]f M m +1 (x) = [x 1 ,...,x m +1 ]f − [x 1 ,...,x m ,x ]f + [x 1 ,...,x m +1 ,x ]f (x − x m +1 ) M m (x) = [x 1 ,...,x m +1 ,x ]f (x m +1 − x) + [x 1 ,...,x m +1 ,x ]f (x − x m +1 ) M m (x) = 0.

Hence the conjecture is true for n = m + 1. Note that to obtain the interpolation polynomial of the next higher degree with New-

ton’s formula, we need only add a term similar to the last term, but involving a new divided difference of one higher order.

In particular, if f ∈ P n , then it follows from (4.2.7) that

[x 1 ,x 2 ,...,x n ,x ]f = 0 ∀x.

For x = x n +1 , this equation is, by Theorem 3.3.4 the only nontrivial relation of the form n

j =1 a j f (x j ) = 0 that holds for all f ∈ P n .

Theorem 4.2.2.

Let x i , i = 1 : n, be pairwise distinct interpolation points. For every n, the divided difference [x 1 ,x 2 ,...,x

n ]f is the unique coefficient of x −1 in the interpolation polynomial p ∈P n . 129 Further, we have

[x 1 ,x 2 ,...,x n ]f =

Proof. The first statement follows from (4.2.3). The right-hand side of (4.2.8) is the coeffi- cients of x n −1 obtained by using Lagrange’s interpolation formula, Theorem

129 Some authors take this as the definition of the divided difference.

4.2. Interpolation Formulas and Algorithms 361 It follows from (4.2.8) that the divided differences are symmetric functions of its

arguments and continuous functions of its arguments, as long as the points x i are distinct and f (x) is continuous.

Assume k > i. By the definition of divided differences, [x i +1 ,...,x k −1 ,x ]f − [x i +1 ,...,x k −1 ,x k ]f

[x i +1 ,...,x k −1 ,x k ,x ]f = .

x −x k

Now set x = x i and use the symmetry property (Theorem 4.2.2). We obtain the formula

i ,x i ,...,x k ,x k +1 ,...,x k −1 ]f − [x i +1 ,...,x k −1 ,x k ]f

This formula can be used recursively to compute the divided differences. The computation is conveniently arranged in the table below for n = 5 (recall that [x i ]f = f (x i ) ).

x 5 [x 5 ]f This table is called a divided-difference table. Each entry in the table is computed from

the two entries above and below in the previous column. Hence the complete table can be constructed, for example, column by column or diagonal by diagonal.

The exact remainder term in Theorem 4.2.1 is not directly useful since unknown value

f (x) occurs in the divided difference [x 1 ,...,x n ,x ]f . We now derive another expression for the remainder term. In the following, int (x, x 1 ,...,x n ) denotes the smallest interval

that contains the points x and x 1 ,...,x n .

Theorem 4.2.3 (The Remainder Term for Interpolation). Let f be a given real function, with f (n) continuous in int (x, x 1 ,x 2 ,...,x n ). Denote by p the polynomial of degree n − 1 for which p(x i ) = f (x i ), i = 1 : n. Then

f (n) (ξ x )

f (x) − p(x) = [x 1 ,x 2 ,...,x n ,x ]f M n (x) =

M n (x), (4.2.10)

2 n M n (x) = i =1 (x −x i ), for some point ξ x ∈ int (x, x 1 ,x 2 ,...,x n ), and

f (n) (ξ )

[x 1 ,x 2 ,...,x n ,x n +1 ]f = , ξ ∈ int (x 1 ,...,x n +1 ). (4.2.11)

Proof. Following a proof due to Cauchy, we introduce a new variable z and set G(z) = f (z) − p(z) − [x 1 ,x 2 ,...,x n ,x ]f M n (z).

362 Chapter 4. Interpolation and Approximation We know by Theorem 4.2.1 that

f (x) − p(x) = [x 1 ,x 2 ,...,x n ,x ]f M n (x), (4.2.12) hence G(x) = 0. Then G(z) = 0 for z = x, x 1 ,x 2 ...,x n . From repeated use of Rolle’s

theorem it follows that there exists a point ξ

x ∈ int (x, x 1 ,x 2 ,...,x n ) such that G (ξ x ) = 0. Since p (n) (z)

(n)

= 0 and M (n) n (z) = n! for all z, G (z) =f (z) − [x 1 ,x 2 ,...,x n ,x ]f n!. If we now put z = ξ x , we obtain

[x 1 ,x 2 ,...,x n ,x ]f =

Put this into the definition of G(z), and set z = x. Since G(x) = 0, the first statement follows. The second statement follows from (4.2.13) for x = x n +1 .

Notice the similarity to the remainder term in Taylor’s formula. Notice also that the right-hand side of (4.2.10) is zero at the grid points—as it should be. In the proof of this theorem we have assumed that x i , x, and f (x) are real. In Sec. 4.3.1 we shall derive a remainder term for the general case that x i are complex interpolation points and also consider the case when the points are allowed to coincide.

Theorem 4.2.4.

For equidistant points x i =x 1 + (i − 1)h, f i = f (x i ), it holds that

[x i ,x i +1 ,...,x i +k ]f = k .

Proof. The proof is obtained by induction, with the use of (4.2.9). The details are left to the reader.

k f n k f We have noted above that in the notation for the equidistant case ∇ (k) ≈h , while in the divided difference notation f [x n ,x n −1 ,...,x n

−k (k) ]≈f /k !. For the basis functions of the interpolation formulas we have, respectively,

x = O(1),

k · · · (x − x n −k+1 ) = O(h ),

(x −x n )(x −x n −1 )

provided that x − x n −j = O(h), j = 0 : k − 1. We are now in a position to give a short proof of the important formula (3.3.4) that

we now formulate as a theorem.

Theorem 4.2.5.

Assume that f ∈ C k . Then

k f =h (k) (ζ ), ζ ∈ [x, x + kh]. (4.2.15) If f ∈ P k , then 4 k f (x) = 0. Analogous results hold, mutatis mutandis, for backward and

4 k f (x)

central differences.

4.2. Interpolation Formulas and Algorithms 363 Proof. Combine the result in Theorem 4.2.4 with (4.2.11), after making appropriate

substitutions.

Example 4.2.1.

Suppose we want to compute by linear interpolation the value f (x) at a point x = x 0 + θh, h = x 1 −x 0 . Since θ(1 − θ) takes on its maximum value 1/4 for θ = 1/2, it

follows from (4.2.10) that for 0 ≤ θ ≤ 1 the remainder satisfies |f (x) − p(x)| ≤ h 2 M

2 / 8, M 2 =

x max ,x |f (x) |. (4.2.16)

∈int [x 0 1 ]

If the values f 0 and f 1 are given to t correct decimal digits, then the roundoff error in evaluating

p(x) = (1 − θ)f 0 + θf 1 ,

2 8 2 , then the total error in p(x) is bounded by 10 −t . This analysis motivates the rule of thumb that linear interpolation suffices if |4 2 f n |/8

is bounded by 2 1 10 −t . Further, if h M 2 ≤ 1 · 10 −t

is a tolerable truncation error. To form the Newton interpolation polynomial we only need one diagonal of the

divided-difference table. It is not necessary to store the entire table. The following al- gorithm replaces (overwrites) the given function values

f i = f (x i ) = [x 1 ]f, i = 1 : n,

by the downward diagonal of divided differences of the divided-difference table,

[x 1 ]f, [x 1 ,x 2 ]f, . . . , [x 1 ,x 2 ,...,x n ]f.

At step j the jth column of the table is computed as follows:

% Compute downward diagonal of the divided-difference % table. Function values are overwritten for j = 1:n-1

for i = n:(-1):j+1

f(i) = (f(i) - f(j-1))/(x(i) - x(i-j)); end end

Note that to avoid overwriting data needed later it is necessary to proceed from the bottom of each column. The algorithm uses n(n −1)/2 divisions and n(n−1) subtractions.

Newton’s interpolation formula has the advantage that if it is not known in advance how many interpolation points are needed to achieve the required accuracy, then one inter- polation point can be added at a time. The following progressive algorithm computes the divided-difference table one diagonal at a time. In the jth step, j = 2 : n, the entries

[x j −1 ,x j ]f, . . . , [x 1 ,x 2 ,...,x j ]f

364 Chapter 4. Interpolation and Approximation on the upward diagonal of the divided-difference table overwrite the function values

f j −1 ,...,f 1 .

% Compute upward diagonal of the divided-difference % table. Function values are overwritten for j = 2:n

for i = j:(-1):2

f(i-1) = (f(i) - f(i-1))/(x(j) - x(i-1)); end end

By Theorem 4.2.1 the Newton polynomial has the form

j & −1

p(x) =c 1 +

(x −x i ), c j = [x 1 ,...,x j ]f. (4.2.17)

j =2

i =1

Substituting z for x we have, by a simple generalization of Horner’s rule, that p(z) = b 1 (z) , where

b n =c n , b i (z) =b i +1 (z)(z −x i ) +c i , i = n − 1 : −1 : 1. (4.2.18) This recursion can be used algebraically or to evaluate p(x) for a given numerical value

x = z. It is straightforward to show that in the latter case the computed result is the exact value corresponding to slightly perturbed divided differences; cf. Problem 2.3.6.

The auxiliary quantities b n (z), . . . , b 2 (z) are of independent interest, since

n −1

p(x) =b 1 + (x − z) b 2 +

b j +1 φ j −1 (x) . (4.2.19)

j =2

Hence they give the coefficients of the quotient polynomial in synthetic division of p(x) with (x − z). The proof of this result is left as an exercise. Derivatives of a Newton

polynomial can be evaluated by repeated applications of the Horner scheme.

Example 4.2.2.

Compute the interpolation polynomial for the following table:

(the entries appearing in the Newton forward interpolation formula are boldface). We get

p(x) = 0 + 2(x − 1) + 1(x − 1)(x − 2) + 0(x − 1)(x − 2)(x − 4)

=x 2 − x.

4.2. Interpolation Formulas and Algorithms 365 Note that for these particular data the unique interpolation polynomial in P 4 actually belongs

to the subspace P 3 .

Example 4.2.3.

Set f (x; z) = 1/(z − x); x is the variable, z is a parameter, and both may be com- plex. The following elementary, though remarkable, expansion can be proved directly by

M n (z)(z − x)

j =1

When we match this with Newton’s interpolation formula we find that

[x 1 ,x 2 ,...,x j ]f (x; z) =

M j (z)

[x 1 ,x 2 ,...,x j ,x ]f (x; z) =

. (4.2.22) M j (z)(z − x)

These formulas can also be proved by induction, by working algebraically with 1/(z −x) in the divided-difference table (Problem 4.2.4). See also Problem 3.3.3 (a) for the equidistant

case. This is more than a particular example. Since 1/(z − x) is the kernel of Cauchy’s

integral (and several other integral representations), this expansion is easily generalized to arbitrary analytic functions; see Sec. 4.3.2.

be distinct. (They are consistent with the extension to nondistinct points that will be made in Sec. 4.3.1.) Everything is continuous except if z = x i , i = 1 : n, or, of course, if z = x. If all the x i coincide, we obtain a geometric series with a remainder.

An interesting feature is that these formulas do not require that the points x i

For given interpolation points the divided differences in Newton’s interpolation for- mula depend on the ordering in which the points x i are introduced. Mathematically all orderings give the same unique interpolation polynomial. But the condition number for the coefficients c in the Newton polynomial can depend strongly on the ordering of the interpolation points . For simple everyday interpolation problems the increasing order

x 1 <x 2 < ···<x n will give satisfactory results. If the point ˜x where the polynomial is to be evaluated is known, then an ordering such that

| ˜x − x 1 | ≤ | ˜x − x 2 | ≤ · · · ≤ | ˜x − x n |

can be recommended. In the equidistant case this corresponds to using Stirling’s interpola- tion formula (3.3.39). In case convergence is slow and an interpolation polynomial of high

366 Chapter 4. Interpolation and Approximation order has to be used, another suitable choice is the Leja ordering, defined by

|x i −x k |, j = 2 : n − 1. (4.2.23)

1≤i≤n

k =1

≥j k =1

Let K be a compact set in the complex plane with a connected complement. Any sequence of points ξ 1 ,ξ 2 ,... which satisfies the conditions

|ξ − ξ k |, j = 2, 3, . . . , (4.2.24)

are Leja points for K. The points may not be uniquely defined by (4.2.24). For a real interval [a, b] the Leja points are distributed similarly to the Chebyshev points. The main advantage of the Leja points is that it is easy to add new Leja points successively to an already computed sequence of Leja points; see Reichel [299].