Distance and Norm

4.5.1 Distance and Norm

For the study of accuracy and convergence of methods of interpolation and approximation we now introduce the concept of a metric space. By this we understand a set of points S and a real-valued function d, a distance defined for pairs of points in S in such a way that the following axioms are satisfied for all x, y, z in S. (Draw a triangle with vertices at the points x, y, z.)

1. d(x, x) = 0

(reflexivity), (positivity),

3. d(x, y) = d(y, x)

(symmetry),

4. d(x, y) ≤ d(x, z) + d(z, y)

(triangle inequality).

The axioms reflect familiar features of distance concepts used in mathematics and everyday life, such as the absolute value of complex numbers, the shortest distance along a geodesic on the surface of the Earth, or the shortest distance along a given road system. 143

Many other natural and useful relations can be derived from these axioms, such as

n −1

d(x, y) ≥ |d(x, z) − d(y, z)|,

d(x 1 ,x n ) ≤

d(x i ,x i +1 ), (4.5.1)

i =1

where x 1 ,x 2 ,...,x n is a sequence of points in S.

Definition 4.5.1.

A sequence of points {x n } in a metric space S is said to converge to a limit x ∗ ∈S if d(x n ,x ∗ ) → 0. As n → ∞, we write x n →x ∗ or lim n →∞ x n =x ∗ . A sequence {x n }

in S is called a Cauchy sequence if for every ǫ > 0 there is an integer N(ǫ) such that d(x m ,x n ) < ǫ for all m, n ≥ N(ǫ). Every convergent sequence is a Cauchy sequence, but the converse is not necessarily true. S is called a complete space if every Cauchy sequence in S converges to a limit in S.

It is well known that R satisfies the characterization of a complete space, but the set of rational numbers is not complete. For example, the Newton iteration x

√ 1 = 1, x n

2, which is not a rational number. Many important problems in pure and applied mathematics can be formulated as minimization problems. The function space terminology often makes proofs and algorithms less abstract.

+1 = 2 (x n + 2/x n ) , defines a sequence of rational numbers that converges to

143 If S is a functions space, the points of S are functions with operands in some other space, for example, in R or R n .

4.5. Approximation and Function Spaces 441 Most spaces that we shall encounter in this book are linear spaces. Their elements

are called vectors, which is why these spaces are called vector spaces. Two operations are defined in these spaces, namely the addition of vectors and the multiplication of a vector by

a scalar. They obey the usual rules of algebra. 144 The set of scalars can be either R or C; the vector space is then called real or complex, respectively.

Definition 4.5.2.

Let f be in a metric space B with a distance function d(x, y), and let S be a subset or linear subspace of B. We define the distance of f to S by

dist (f, S) = inf d(f, g).

g ∈S

Much of our discussion also applies to linear spaces of infinite dimension, i.e., func- tion spaces . The elements (vectors) then are functions of one or several real variables on

a compact set, i.e., a closed bounded region. The idea of such a functions space is now illustrated in an example.

Example 4.5.1.

Consider the set of functions representable by a convergent power series on the interval [−1, 1],

f (t )

=c 2

0 +c 1 t +c 2 t +···.

This is an infinite-dimensional linear space. The functions 1, t, t 2 ,... can be considered as

a standard basis of this space. The coordinates of f (t) then are the vector c 0 ,c 1 ,c 2 ,... . We shall be concerned with the problem of linear approximation, i.e., a function f

is to be approximated using a function f ∗ that can be expressed as a linear combination of n

given linearly independent functions φ 1 (x), φ 2 (x), . . . , φ n (x) ,

f ∗ =c 1 φ 1 (x) +c 2 φ 2 (x) +···+c n φ n (x), (4.5.3) where c 1 ,c 2 ,...,c n are parameters to be determined. 145 They may be considered as coor-

dinates of f ∗ in the functions space spanned by φ 1 (x), φ 2 (x), . . . , φ n (x) . In a vector space the distance of the point f from the origin is called the norm of f

the relevant space. The definition of the norm depends on the space. The following axioms must be satisfied.

Definition 4.5.3.

1–3 below for all f, g ∈ S, and for all scalars λ.

144 See Sec. A.1 in Online Appendix A for a summary about vector spaces. In more detailed texts on linear algebra or functional analysis you find a collection of eight axioms (commutativity, associativity, etc.) required by a linear

vector space. 145 The functions φ j , however, are typically not linear. The term “linear interpolation” is, from our present point

of view, rather misleading.

442 Chapter 4. Interpolation and Approximation (positivity),

(homogeneity), (triangle inequality).

A normed vector space is a metric space with the distance

d(x, y)

If it is also a complete space, it is called a Banach space. 146 The most common norms in spaces of (real and complex) infinite sequences x =

(ξ 1 ,ξ 2 ,ξ 3 , . . .) T or spaces of functions on a bounded and closed interval [a, b] are ∞ = max j |ξ j |,

∞ = max x |f (x)|, ∈[a,b]

2 2,[a,b] =

|f (x)| dx ,

j =1 a ∞ b

1 = |ξ j |,

1 1,[a,b] =

|f (x)| dx.

j =1

These norms are called • the ℓ ∞ or max(imum) norm (or uniform norm);

• the Euclidean norm (or the l 2 -norm for coordinate sequences and L 2 norm for inte- grals);

• the ℓ 1 -norm. It is possible to introduce weight function w(x), which is continuous and strictly

positive on the open interval (a, b), into these norms. For example,

= 2 w j 2 |ξ j | , 2,w = |f (x)| w(x) dx ,

j =1

is the weighted Euclidean norm. We assume that the integrals

|x| k w(x) dx

exist for all k. Integrable singularities at the endpoints are permitted; an important example is w(x) = (1 − x 2 ) −1/2 on the interval [−1, 1].

146 Stefan Banach (1892–1945), a Polish mathematician at the University in Lvov. Banach founded modern functional analysis and made major contributions to the theory of topological vector spaces, measure theory, and

related topics. In 1939 he was elected President of the Polish Mathematical Society.

4.5. Approximation and Function Spaces 443 The above norms are special cases or limiting cases (p → ∞ gives the max norm)

of the l p - or L p -norms and weighted variants of these. They are defined for p ≥ 1 as follows: 147

|f (x)| dx . (4.5.4)

j =1

(The sum in the l p -norm has a finite number of terms, if the space is finite dimensional.) Convergence in a space, equipped with the max norm, means uniform convergence. Therefore, the completeness of the space C[a, b] follows from a classical theorem of analysis that tells us that the limit of a uniformly convergent sequence is a continuous function. The generalization of this theorem to several dimensions implies the completeness of the space of continuous functions, equipped with the max norm on a closed bounded region in R n .

Other classes of functions can be normed with the max norm max x ∈[a,b] |f (x)|, for example, C 1 [a, b]. This space is not complete, but one can often live well with incomplete- ness. The notation L 2 -norm comes from the function space L 2

b [a, b], which is the class of functions for which the integral a |f (x)| 2 dx exists, in the sense of Lebesgue. The

Lebesgue integral is needed in order to make the space complete and greatly extends the scope of Fourier analysis. No knowledge of Lebesgue integration is needed for the study of this book, but this particular fact can be interesting as background. One can apply this norm also to the (smaller) class of continuous functions on [a, b]. In this case the Riemann integral is equivalent. This also yields a normed linear space but it is not complete. 148

Although C[0, 1] is an infinite-dimensional space, the restriction of the continuous functions f to the equidistant grid defined by x i = ih, h = 1/n, i = 0 : n, constitutes

an (n + 1)-dimensional space, with the function values on the grid as coordinates. If we choose the norm

|f (x)| dx

Limit processes of this type are common in numerical analysis. Notice that even if n + 1 functions φ 1 (x), φ 1 (x), . . . , φ n +1 (x) are linearly indepen-

dent on the interval [0, 1] (say), their restrictions to a grid with n points must be linearly dependent; but if a number of functions are linearly independent on a set M (a discrete set

or continuum), any extensions of these functions to a set M ′ ⊃ M will also be linearly independent.

p is derived from two classical inequalities due to Hölder and Minkowski. Elegant proofs of these are found in Hairer and Wanner [178, p. 327].

A modification of the L 2 -norm that also includes higher derivatives of f is used in the Sobolev spaces, which are used as a theoretical framework for the study of the practically very important finite element methods (FEMs), used in particular for the numerical treatment of partial differential equations.

444 Chapter 4. Interpolation and Approximation The class of functions, analytic in a simply connected domain D ⊂ C, normed with

D = max z ∈∂D |f (z)|, is a Banach space denoted by Hol(D). (The explanation of this term is that analytic functions are also called holomorphic.) By the maximum principle for

D for z ∈ D.