Properties of Difference Operators

3.3.1 Properties of Difference Operators

Difference operators are handy tools for the derivation, analysis, and practical application of numerical methods for many problems for interpolation, differentiation, and quadrature of a function in terms of its values at equidistant arguments. The simplest notations for difference operators and applications to derivatives were mentioned in Sec. 1.1.4.

Let y denote a sequence {y n }. Then we define the shift operator E (or translation operator) and the forward difference operator 4 by the relations

Ey = {y n +1 },

4y = {y n +1 −y n };

E and 4 are thus operators which map one sequence to another sequence. Note, however,

that if y n is defined for a ≤ n ≤ b only, then Ey b is not defined, and the sequence Ey has fewer elements than the sequence y. (It is therefore sometimes easier to extend the sequences to infinite sequences, for example, by adding zeros in both directions outside the original range of definition.)

These operators are linear, i.e., if α, β are real or complex constants and if y, z are two sequences, then E(αy + βz) = αEy + βEz, and similarly for 4.

Powers of E and 4 are defined recursively, i.e.,

= E(E k −1 y), 4 k y = 4(4 −1 y). By induction, the first relation yields E k y = {y n +k }. We extend the validity of this relation

to k = 0 by setting E 0 y = y and to negative values of k. 4 k y is called the kth difference of the sequence y. We make the convention that 4 0 = 1. There will be little use of 4 k for

negative values of k in this book, although 4 −1 can be interpreted as a summation operator. Note that 4y = Ey − y, and Ey = y + 4y for any sequence y. It is therefore

convenient to express these as equations between operators:

4 = E − 1,

E = 1 + 4.

The identity operator is in this context traditionally denoted by 1. It can be shown that all formulas derived from the axioms of commutative algebra can be used for these operators, for example, the binomial theorem for positive integral k,

kkk

4 j = (E − 1) = ( −1) −j E , E = (1 + 4) = 4 , (3.3.1)

j =0

j =0

3.3. Difference Operators and Operator Expansions 221 giving

(4 y) n . (3.3.2)

We abbreviate the notation further and write, for example, Ey n =y n +1 instead of (Ey) n = y n +1 , and 4 k y n instead of (4 k y) n . But it is important to remember that 4 operates on sequences and not on elements of sequences. Thus, strictly speaking, this abbreviation is incorrect, though convenient. The formula for E k will, in the next subsection, be extended to an infinite series for nonintegral values of k, but that is beyond the scope of algebra.

A difference scheme consists of a sequence and its difference sequences, arranged in the following way:

y 0 4y 0 y 1 4 2 y 0 4y 1 4 3 y 0 y 2 4 2 y 1 4 4 y 0 4y 2 4 3 y 1 y 3 4 2 y 2 4y 3 y 4

A difference scheme is best computed by successive subtractions; the formulas in (3.3.1) are used mostly in theoretical contexts. In many applications the quantities y n are computed in increasing order n = 0, 1, 2, . . . , and it is natural that a difference scheme is constructed by means of the quantities previously computed. One therefore introduces the backward difference operator

∇y n =y n −y n −1 = (1 − E −1 )y n .

For this operator we have ∇ k = (1 − E −1 ) k ,

(3.3.3) Note the reciprocity in the relations between ∇ and E −1 .

E −k

= (1 − ∇) k .

Any linear combination of the elements y n ,y n −1 ,...,y n −k can also be expressed as

a linear combination of y n n ,..., k ∇ y n , and vice versa. , ∇y 68 For example, y n +y n −1 +y n −2 = 3y n − 3∇y n

+∇ 2 y n , because 1 + E 2 −1 +E −2

= 1 + (1 − ∇) + (1 − ∇) 2 = 3 − 3∇ + ∇ . By reciprocity, we also obtain y n + ∇y n

+∇ 2 y n = 3y n − 3y n

−1 +y n −2 .

68 An analogous statement holds for the elements y n ,y n +1 ,...,y n +k and forward differences.

222 Chapter 3. Series, Operators, and Continued Fractions In this notation the difference scheme reads as follows.

y 0 ∇y 1

y 1 ∇ 2 y 2 ∇y 2 ∇ 3 y 3 y 2 ∇ 2 y 3 ∇ 4 y 4 ∇y 3 ∇ 3 y 4 y 3 ∇ 2 y 4 ∇y 4 y 4

In the backward difference scheme the subscripts are constant along diagonals directed upward (backward) to the right, while in the forward difference scheme subscripts are constant along diagonals directed downward (forward). Note, for example, that ∇ k y n =

4 k y n −k . In a computer, a backward difference scheme is preferably stored as a lower triangular matrix.

Example 3.3.1.

Part of the difference scheme for the sequence y = {. . . , 0, 0, 0, 1, 0, 0, 0, . . .} is given below.

This example shows the effect of a disturbance in one element on the sequence of the higher differences. Because the effect broadens out and grows quickly, difference schemes are useful in the investigation and correction of computational and other errors, so-called difference checks . Notice that, since the differences are linear functions of the sequence,

a superposition principle holds. The effect of errors can thus be estimated by studying simple sequences such as the one above.

3.3. Difference Operators and Operator Expansions 223

Example 3.3.2.

The following is a difference scheme for a five-decimal table of the function f (x) = tan x, x ∈ [1.30, 1.36], with step h = 0.01. The differences are given with 10 −5 as unit.

1.36 4.67344 We see that the differences decrease roughly by a factor of 0.1—that indicates that the step

size has been chosen suitably for the purposes of interpolation, numerical quadrature, etc. until the last two columns, where the rounding errors of the function values have a visible effect.

Example 3.3.3.

For the sequence y n

= (−1) n one finds easily that

n , = 2y 2 ∇ y n = 4y n ,..., ∇ y n =2 y n . If the errors in the elements of the sequence are bounded by ǫ, it follows that the errors of

∇y k

the kth differences are bounded by 2 k ǫ . A rather small reduction of this bound is obtained if the errors are assumed to be independent random variables (cf. Problem 3.4.24).

It is natural also to consider difference operations on functions not just on sequences.

E and 4 map the function f onto functions whose values at the point x are

E f (x) = f (x + h), 4f (x) = f (x + h) − f (x), where h is the step size. Of course, 4f depends on h; in some cases this should be indicated in

the notation. One can, for example, write 4 h f (x) , or 4f (x; h). If we set y n = f (x 0 + nh), the difference scheme of the function with step size h is the same as for the sequence {y n }.

Again it is important to realize that, in this case, the operators act on functions, not on the values of functions. It would be more correct to write f (x 0 + h) = (Ef )(x 0 ) . Actually, the notation (x 0 )Ef would be even more logical, since the insertion of the value of the argument x 0 is the last operation to be done, and the convention for the order of execution of operators proceeds from right to left. 69

69 The notation [x 0 ]f occurs, however, naturally in connection with divided differences; see Sec. 4.2.1.

224 Chapter 3. Series, Operators, and Continued Fractions Note that no new errors are introduced during the computation of the differences,

but the effects of the original irregular errors, for example, rounding errors in y, grow exponentially . Note that systematic errors, for example, truncation errors in the numerical solution of a differential equation, often have a smooth difference scheme. For example, if the values of y have been produced by the iterative solution of an equation, where x is a parameter, with the same number of iterations for every x and y and the same algorithm for the first approximation, then the truncation error of y is likely to be a smooth function of x.

Difference operators are in many respects similar to differentiation operators. Let f

be a polynomial. By Taylor’s formula, 4f (x) = f (x + h) − f (x) = hf ′

2 +···. We see from this that deg 4f = deg f − 1. Similarly, for differences of higher order, if f

(x)

+ 2 h f ′′ (x)

is a polynomial of degree less than k, then

4 p f (x) = 0 ∀p ≥ k. The same holds for backward differences.

4 k −1 f (x) = constant,

The following important result can be derived directly from Taylor’s theorem with the integral form of the remainder. Assume that all derivatives of f up to kth order are continuous. If f ∈ C k ,

=h k f (k) (ζ ), ζ ∈ [x, x + kh]. (3.3.4) Hence h −k 4 k f (x) is an approximation to f (k) (x) ; the error of this approximation ap-

4 k f (x)

proaches zero as h → 0 (i.e., as ζ → x). As a rule, the error is approximately proportional to h. We postpone the proof to Sec. 4.2.1, where it appears as a particular case of a theorem

concerning divided differences. Even though difference schemes do not have the same importance today that they had in the days of hand calculations or calculation with desk calculators, they are still important conceptually, and we shall also see how they are still useful in practical computing. In a computer it is more natural to store a difference scheme as an array, with y n

, ∇y 2 n ,∇ y n ,..., ∇ k y n in a row (instead of along a diagonal). Many formulas for differences are analogous to formulas for derivatives, though usually more complicated. The following results are among the most important.

Lemma 3.3.1.

It holds that

k ∇ x (a ) = (1 − a −h ) a . (3.3.5) For sequences, i.e., if h = 1,

4 k (a x )

h = (a k − 1) a x ,

{a n } = (a − 1) {a }, 4 {2 } = {2 }. (3.3.6) Proof. Let c be a given constant. For k = 1 we have

h − ca x = c(a − 1)a . The general result follows easily by induction. The backward difference formula is derived

4(ca x )

= ca x +h

− ca x = ca a h x

in the same way.

3.3. Difference Operators and Operator Expansions 225 Lemma 3.3.2 (Difference of a Product).

(3.3.7) Proof. We have

4(u n v n ) =u n 4v n + 4u n v n +1 .

4(u n v n ) =u n +1 v n +1 −u n v n =u n (v n +1 −v n ) + (u n +1 −u n )v n +1 .

Compare the above result with the formula for differentials, d(uv) = udv + vdu. Note that we have v n +1 (not v n ) on the right-hand side.

Lemma 3.3.3 (Summation by Parts).

N −1 −1

u n 4v n =u N v N −u 0 v 0 −

4u n v n +1 . (3.3.8)

n =0

n =0

Proof. (Compare with the rule for integration by parts and its proof!) Notice that N −1

Use this on w n =u n v n . From the result in Lemma 3.3.1 one gets after summation

and the result follows. (For an extension, see Problem 3.3.2 (d).)