Mathematical Properties of Orthogonal Polynomials

4.5.5 Mathematical Properties of Orthogonal Polynomials

By a family of orthogonal polynomials we mean a triangle family of polynomials (see (4.1.8)), which (in the continuous case) is an orthogonal system with respect to a given inner product. The theory of orthogonal polynomials is also of fundamental importance for many problems which at first sight seem to have little connection with approximation (e.g., numerical integration, continued fractions, and the algebraic eigenvalue problem).

We assume in the following that in the continuous case the inner product is

(f, g) =

f (x)g(x)w(x) dx, w(x) ≥ 0, (4.5.30)

where −∞ ≤ a < b ≤ ∞. We assume that the weight function w(x) ≥ 0 is such that the

x k w(x) dx

0. In the discrete case, we define the weighted discrete inner product of two real-valued functions f and g on the grid {x m

are defined for all k ≥ 0, and µ 0 >

i } j =0 of distinct points by

(f, g) =

w i f (x i )g(x i ), w i > 0. (4.5.32)

i =0

Note that both these inner products have the property that

(xf, g) = (f, xg).

458 Chapter 4. Interpolation and Approximation The continuous and discrete case are both special cases of the more general inner product

(f, g) =

f (x)g(x) dα(x),

where the integral is a Stieltjes integral (see Definition 3.4.4) and α(x) is allowed to be discontinuous. However, in the interest of clarity, we will in the following treat the two cases separately.

The weight function w(x) determines the orthogonal polynomials φ n (x) up to a con- stant factor in each polynomial. The specification of those factors is referred to as stan- dardization . These polynomials satisfy a number of relationships of the same general form. In the case of a continuously differentiable weight function w(x) we have an explicit expression

φ n (x) = n n {w(x)(g(x)) }, n = 0, 1, 2, . . . , (4.5.35)

a n w(x) dx

where g(x) is a polynomial in x independent of n. This is Rodrigues’ formula. The orthogonal polynomials also satisfy a second order differential equation,

(4.5.36) where g 2 (x) and g 1 (x) are independent of n and a n is a constant only dependent on n.

g 2 (x)φ ′′ n +g 1 (x)φ ′ n +a n φ n = 0,

Let p n (x) =k n x n + · · · , n = 0, 1, 2, . . . , be a family of real orthogonal polynomials. The symmetric function

K n (x, y) =

p k (x)p k (y)

k =0

is called the kernel polynomial of order n for the orthogonal system. It can be shown that the kernel polynomial has the reproducing property that for every polynomial p of degree at most n

(4.5.38) Here the subscript x indicates that the inner product is taken with respect to x. Conversely,

(p(x), K n (x, y) x ) = p(y).

if K(x, y) is a polynomial of degree at most n in x and y and if (p(x), K(x, y) x ) = p(y), for all polynomials p of degree at most n, then K(x, y) = K n (x, y) .

An alternative expression, the Christoffel–Darboux formula, can be given for the kernel polynomial.

Theorem 4.5.18.

Let p n (x) =k n x n + · · ·, n = 0, 1, 2, . . . , be real orthonormal polynomials. Then

K (x, y) +1 (x)p n (y) −p n (x)p n +1 (y)

Proof. See Davis [92, Theorem 10.1.6]. Given a linearly independent sequence of vectors, an orthogonal system can be derived

by a process analogous to Gram–Schmidt orthogonalization.

4.5. Approximation and Function Spaces 459

Theorem 4.5.19.

For every weight function in an inner product space there is a triangle family of orthogonal polynomials φ k (x), k = 0, 1, 2, . . . , such that φ k (x) has exact degree k, and is orthogonal to all polynomials of degree less than k. The family is uniquely determined apart from the fact that the leading coefficients can be given arbitrary positive values.

The monic orthogonal polynomials satisfy the three-term recurrence formula,

φ k +1 (x) = (x − β k )φ k (x)

k −1 φ k −1 (x), k ≥ 1, (4.5.40) with initial values φ −1 (x) = 0, φ 0 (x) = 1. The recurrence coefficients are given by

Darboux’s formulas

Proof. The proof is by induction. We have φ −1 = 0, φ 0 = 1. Suppose that φ j been constructed for 0 ≤ j ≤ k, k ≥ 0. We now seek a polynomial φ k +1 of degree k + 1

with leading coefficient equal to 1 which is orthogonal to all polynomials of degree ≤ k. k Since {φ j } j =0 is a triangle family, every polynomial of degree k can be expressed as a linear combination of these polynomials. Therefore, we can write

where φ k +1 has leading coefficient one. The orthogonality condition is fulfilled if and only if

j 2 = (xφ k ,φ j ) . This determines the coefficients uniquely. From the definition of inner product (4.5.30), it follows that

k,j

(xφ k ,φ j ) = (φ k , xφ j ).

But xφ j is a polynomial of degree j + 1. Thus if j < k, then it is orthogonal to φ k . So

c kj = 0 for j < k − 1. From (4.5.42) it then follows that

(4.5.43) with c k,k −1 = 0 if k = 0. This has the same form as the original assertion of the theorem if

In the discrete case the division in (4.5.44) can always be performed, as long as k ≤ m. In the continuous case, no reservation need be made.

460 Chapter 4. Interpolation and Approximation The expression for γ 2 k −1 can be written in another way. If we take the inner product

of (4.5.42) and φ k +1 we get

Thus (φ k +1 , xφ k )

, or if we decrease all indices by one, (φ k , xφ k −1 ) k 2 . Substituting this in the expression for γ 2 k −1 gives the second equation of (4.5.41).

If the weight distribution w(x) is symmetric about β, i.e., (in the continuous case) w(β − x) = w(x + β), then β k = β for all k ≥ 0. Further,

k (x + β), k ≥ 0; (4.5.45) that is, φ k is symmetric about β for k even and antisymmetric for k odd. The proof is by

φ k (β

− x) = (−1) k φ

induction. We have φ 0 = 1 and φ 1 (x) =x−β 0 . Clearly (φ 1 ,φ 0 ) = 0 implies that φ 1 is antisymmetric about β and therefore β 0 = β. Thus the hypothesis is true for k ≤ 1. Now assume that (4.5.45) holds for k ≤ n. Then

n 2 + β. Here the first term is zero since it is an integral of an antisymmetric function. It follows that

n −1 φ n −1 (x), which shows that (4.5.45) holds for k = n + 1. An analog result holds for a symmetric

φ n +1 (x) = (x − β)φ n (x)

discrete inner product. Often it is more convenient to consider corresponding orthonormal polynomials ˆφ k (x)

0 √ µ 0 ,µ 0 0 = 1/ 2 2 , and scale the monic orthogonal polynomials according to

(4.5.46) then we find using (4.5.41) that

φ k = (γ 1 ···γ k −1 )ˆ φ k , k ≥ 1;

1 ···γ k −1 k −1

Substituting (4.5.46) in (4.5.40) we obtain the recurrence relation for the orthonormal poly- nomials

(4.5.47) where γ k

γ k ˆφ k +1 (x) = (x − β k )ˆ φ k (x) −γ k −1 ˆφ k −1 (x), k ≥ 1,

k +1

Perhaps the most important example of a family of orthogonal polynomials is the Chebyshev polynomials T n (x) = cos(n arccos(x)) introduced in Sec. 3.2.3. These are orthogonal on [−1, 1] with respect to the weight function (1 − x 2 ) −1/2 and also with respect to a discrete inner product. Their properties can be derived by rather simple methods.

4.5. Approximation and Function Spaces 461

Theorem 4.5.20.

The Chebyshev polynomials have the following two orthogonality properties. Set

(f, g)

= 2 f (x)g(x)( 1−x ) −1/2 dx

(the continuous case). Then (T 0 ,T 0 ) = π, and %

be the zeros of T m +1 (x) and set m π

(f, g) =

f (x k )g(x k ), x k = cos

(the discrete case). Then (T 0 ,T 0 ) = m + 1, and %

dx = sin φdφ = (1 − x 2 ) 1/2 dφ . Hence

(T j ,T k ) = cos jx cos kx dx =

cos(j − k)x + cos(j + k)x dx

1 sin(j + k)π

whereby orthogonality is proved. In the discrete case, set h = π/(m + 1), x µ = h/2 + µh,

(T j ,T k ) = cos jx µ cos kx µ = cos(j − k)x µ + cos(j + k)x µ .

Using notation from complex numbers (i = −1) we have

i(j

Re (T i(j j ,T k ) =

e −k)h(1/2+µ)

e +k)h(1/2+µ) . (4.5.52)

The sums in (4.5.52) are geometric series with ratios e i(j −k)h and e i(j +k)h , respectively. If j

2m

0 < | (j ± k)h| ≤

m +1

462 Chapter 4. Interpolation and Approximation Using the formula for the sum of a geometric series the first sum in (4.5.52) is

i(j −k)(h/2) e i(j −k)(m+1)h −1

i(j

e −k)π −1

e i(j −k)h

e i(j −k)(h/2) −e −i(j−k)(h/2) −1

−1) j −k −1

2i sin(j − k)h/2 .

The real part of the last expression is clearly zero. An analogous computation shows that the real part of the other sum in (4.5.52) is also zero. Thus the orthogonality property holds in the discrete case also. It is left to the reader to show that the expressions when j = k given in the theorem are correct.

For the uniform weight distribution w(x) = 1 on [−1, 1] the relevant orthogonal poly- nomials are the Legendre polynomials 152 P n (x) . The Legendre polynomials are defined

by Rodrigues’ formula

Since (1 − x 2 ) n is a polynomial of degree 2n, P n (x) is a polynomial of degree n. The Legendre polynomials P n =A n x n + · · · have leading coefficient and norm

This standardization corresponds to setting P n ( 1) = 1 for all n ≥ 0. The extreme values are

|P n (x) | ≤ 1, x ∈ [−1, 1].

There seems to be no easy proof for the last result; see [193, p. 219]. Since the weight distribution is symmetric about the origin, these polynomials have the symmetry property

n −x) = (−1) P n (x).

The Legendre polynomials satisfy the three-term recurrence formula P 0 (x) = 1, P 1 (x) = x,

P n (x), n ≥ 1. (4.5.55)

n +1

The first few Legendre polynomials are

2 (x) = ( 3x − 1),

P 3 (x) = ( 5x − 3x),

A graph of the Legendre polynomial P 21 (x) is shown in Figure 4.5.1.

152 Legendre had obtained these polynomials in 1784–1789 in connection with his investigation concerning the attraction of spheroids and the shape of planets.

4.5. Approximation and Function Spaces 463

Figure 4.5.1. The Legendre polynomial P 21 . Often a more convenient standardization is to consider monic Legendre polynomials,

with leading coefficient equal to one. These satisfy P 0 (x) = 1, P 1 (x) = x, and the recurrence formula

P n +1 (x) = xP n (x) − 4n 2 P n −1 (x), n ≥ 1; (4.5.56)

note that we have kept the same notation for the polynomials. It can be shown that P n

n(n − 1)

2(2n − 1) . The Jacobi polynomials 153 J n (x ; α, β) arise from the weight function

=x +c n x n −2 +···, c n =−

1 + x) β , x ∈ [−1, 1], α, β > −1. They are special cases of Gauss’ hypergeometric function F (a, b, c : x),

w(x) = (1 − x) α (

F( −n, α + 1 + β + n, α + 1; x),

(see (3.1.16)). The Jacobi polynomials are usually standardized so that the coefficient A n of x n in J n (x ; α, β) is given by

The Legendre polynomials are obtained as the special case when α = β = 0. The case √ α = β = −1/2, which corresponds to the weight function w(x) = 1/ 1−x 2 , gives the Chebyshev polynomials.

153 Carl Gustav Jacob Jacobi (1805–1851) was a German mathematician. Jacobi joined the faculty of Berlin University in 1825. Like Euler, he was a proficient calculator who drew a great deal of insight from immense

algorithmic work.

464 Chapter 4. Interpolation and Approximation The generalized Laguerre polynomials L (α) n (x) are orthogonal with respect to the

weight function w(x)

=x α e −x , x ∈ [0, ∞], α> −1. ( Setting α = 0, we get the Laguerre polynomials L 0)

n (x) =L n (x) . Standardizing these so that L n ( 0) = 1, they satisfy the three-term recurrence relation

(4.5.57) Rodrigues’ formula becomes

(n + 1)L n +1 (x) = (2n + 1 − x)L n (x) − nL n −1 (x),

n (x) =

(x −α e −x ).

n !x (α) dx n The Hermite polynomials are orthogonal with respect to the weight function w(x) 2 =e −x , −∞ < x < ∞.

With the classic standardization they satisfy the recurrence relation H 0 (x) = 1, H 1 (x) = 2x,

H n +1 (x) = 2xH n (x) − 2nH n −1 (x),

and

−1) H ( 2m)!/m! if n = 2m,

if n = 2m + 1. The Hermite polynomials can also be defined by Rodrigues’ formula:

H n (x) = (−1) e

{e −x 2 }.

dx n

It can be verified that these polynomials are identical to those defined by the recurrence relation.

The properties of some important families of orthogonal polynomials are summarized in Table 4.5.1. Note that here the coefficients in the three-term recurrence relation are given for the monic orthogonal polynomials; cf. (4.5.40).

For equidistant data, the Gram polynomials {P m n,m } n =0 are of interest. 154 These polynomials are orthogonal with respect to the discrete inner product

(f, g) = (1/m)

f (x i )g(x i ), x i = −1 + (2i − 1)/m.

i =1

The weight distribution is symmetric around the origin α k = 0. For the monic Gram polynomials the recursion formula is (see [16])

P −1,m (x) = 0,

P 0,m = 1,

P n +1,m (x) = xP n,m (x) −β n,m P n −1,m (x), n = 0 : m − 1,

154 Jørgen Pedersen Gram (1850–1916), a Danish mathematician, graduated from Copenhagen University and worked as a director for a life insurance company. He introduced the Gram determinant in connection with his

study of linear independence, and his name is also associated with Gram–Schmidt orthogonalization.

4.5. Approximation and Function Spaces 465

Table 4.5.1. Weight functions and recurrence coefficients for some classical monic orthogonal polynomials.

[a, b] w(x)

Orthog. pol.

k 2 [−1, 1]

1 P n (x) Legendre 2 0 4k 2

( 1−x ) −1/2

T n (x) Cheb. 1st π

0 1 2 (k = 1)

4 (k > 1) [−1, 1] 2 ( 1−x ) 1/2

U n (x) Cheb. 2nd π/2

1 − x) β ( 1 + x) J n (x ; α, β) Jacobi [0, ∞]

x α e −x , α > −1 L (α) n (x) Laguerre Ŵ(1 + α) 2k + α + 1 k(k + α)

−x e 2 H n (x) Hermite

0 [−∞, ∞] 1 2 k

where (n < m) and

β n,m =

4n 2 −1 1− m 2 .

When n ≪ m 1/2 , these polynomials are well behaved. But when n ≥ m 1/2 , the Gram polynomials have very large oscillations between the grid points, and a large maximum

norm in [−1, 1]. This fact is related to the recommendation that when fitting a polynomial to equidistant data, one should never choose n larger than about 2m 1/2 .

Complex Orthogonal Polynomials

So far we have considered the inner products (4.5.30) defined by an integral over the real interval [a, b]. Now let Ŵ be a rectifiable curve (i.e., a curve of finite length) in the complex

plane. Consider the linear space of all polynomials with complex coefficients and z = x+iy on Ŵ. Let α(s) be a function on Ŵ with infinitely many points of increase and define an

inner product by the line integral

(p, q) =

p(z)q(z)w(s) d(s).

The complex monomials 1, z, z 2 ,... are independent functions, since if a 0 +a 1 z +a 2 z 2 + ···+a n z n ≡ 0 on Ŵ it would follow from the fundamental theorem of algebra that a 0 =

a 1 =a 2 =···=a n = 0. There is a unique infinite sequence of polynomials φ j = z j +c 1 z j −1 +···+c j , j = 0, 1, 2, . . . , which are orthonormal with respect to (4.5.58). They can be constructed by Gram–Schmidt orthogonalization as in the real case.

466 Chapter 4. Interpolation and Approximation An important case is when Ŵ is the unit circle in the complex plane. We then write

(p, q) =

p(z)q(z) dα(t ), z

where the integral is to be interpreted as a Stieltjes integral. The corresponding orthog- onal polynomials are known as Szeg˝o polynomials and have applications, e.g., in signal processing.

Properties of Szeg˝o polynomials are discussed in [174]. Together with the reverse polynomials ˜φ

j (z) =z j φ j ( 1/z) they satisfy special recurrence relations. A linear combi- nation j =0 c j φ j (z) can be evaluated by an analogue to the Clenshaw algorithm; see [5].