Complete Monotonicity and Related Concepts

3.4.4 Complete Monotonicity and Related Concepts

For the class of completely monotonic sequences and some related classes of analytic func- tions the techniques of convergence acceleration can be put on a relatively solid theoretical basis.

Definition 3.4.2.

A sequence {u n } is completely monotonic (c.m.) for n ≥ a if and only if

u n ≥ 0, (−4) j u n ≥ 0 ∀j ≥ 0, n ≥ a (integers). Such sequences are also called totally monotonic. The abbreviation c.m. will be used,

both as an adjective and as a noun, and both in singular and in plural. The abbreviation d.c.m. will similarly be used for the difference between two completely monotonic sequences. (These abbreviations are not generally established.)

A c.m. sequence {u n } ∞ 0 is minimal if and only if it ceases to be a c.m. if u 0 is decreased while all the other elements are unchanged. This distinction is of little importance to us, since

we usually deal with a tail of some given c.m. sequence, and it can be shown that if {u n } ∞ 0 is c.m., then {u n } ∞ 1 is a minimal c.m. sequence . Note that, e.g., the sequence {1, 0, 0, 0, . . .} is a nonminimal c.m., while {0, 0, 0, 0, . . .} is a minimal c.m. Unless it is stated otherwise we shall only deal with minimal c.m. without stating this explicitly all the time.

Definition 3.4.3.

A function u(s) is c.m. for s ≥ a, s ∈ R, if and only if u(s)

≥ 0, (−1) (j ) u (j ) (s) ≥ 0, s ≥ a ∀ j ≥ 0 (integer), ∀ s ≥ a (real). u(s) is d.c.m. if it is a difference of two c.m. on the same interval.

3.4. Acceleration of Convergence 285 We also need variants with an open interval. For example, the function u(s) = 1/s is

c.m. in the interval [a, ∞) for any positive a, but it is not c.m. in the interval [0, ∞]. The simplest relation of c.m. functions and c.m. sequences reads as follows: if the

function u(s) is c.m. for s ≥ s 0 , then the sequence defined by u n = u(s 0 + hn), (h > 0), n

= 0, 1, 2, . . . , is also c.m. since, by (3.3.4), (−4) j = (−hD) u(ξ ) ≥ 0 for some ξ ≥s 0 .

A function is absolutely monotonic in an (open or closed) interval if the function and all its derivatives are nonnegative there. The main reason why the analysis of a numerical method is convenient for c.m. and d.c.m. sequences is that they are “linear combinations of exponentials,” according to the theorem below. The more precise meaning of this requires the important concept of a Stieltjes integral . 96

Definition 3.4.4.

The Stieltjes integral b

a f (x) dα(x) is defined as the limit of sums of the form

f (ξ i ) α(x i +1 ) − α(x i ) , ξ i ∈ [x i ,x i +1 ], (3.4.23)

where

a =x 0 <x 1 <x 2 < ···<x N =b

is a partition of [a, b]. Here f (x) is bounded and continuous, and α(x) is of bounded vari- ation in [a, b], i.e., the difference between two nondecreasing and nonnegative functions.

The extension to improper integrals where, for example, b = ∞, α(b) = ∞, is made in a similar way as for Riemann or Lebesgue integrals. The Stieltjes integral is much

used also in probability and mechanics, since it unifies the treatment of continuous and discrete (and mixed) distributions of probability or mass. If α(x) is piecewise differentiable, then dα(x) = α ′

(x) dx b , and the Stieltjes integral is simply

a f (x)α ′ (x) dx . If α(x) is

a step function, with jumps (also called point masses) m i at x = x i , i = 1 : n, then dα(x i ) = lim ǫ ↓0 α(x i + ǫ) − α(x i − ǫ) = m i ,

f (x) dα(x) =

m i f (x i ).

a i =1

(It has been assumed that f (x) is continuous at x i , i = 1 : n. Integration by parts is as usual; the following example is of interest to us. Suppose that α(0) = 0, α(x) = o(e cx ) as x → ∞, and that ℜs ≥ c. Then

e −sx dα(x) =s

α(x)e −sx dx.

96 Thomas Jan Stieltjes (1856–1894) was born in the Netherlands. After working with astronomical calculations at the observatory in Leiden, he accepted a position in differential and integral calculus at the University of

Toulouse, France. He did important work on continued fractions and the moment problem, and invented a new concept of the integral.

286Chapter 3. Series, Operators, and Continued Fractions The integral on the left side is called a Laplace–Stieltjes transform, while the integral on

the right side is an ordinary Laplace transform. Many properties of power series, though not all, can be generalized to Laplace–Stieltjes integrals—set z = e −s . Instead of a disk

of convergence, the Laplace–Stieltjes integral has a (right) half-plane of convergence. A difference is that the half-plane of absolute convergence may be different from the half-plane of convergence.

We shall be rather brief and concentrate on the applicability to the study of numerical methods. We refer to Widder [373, 374] for proofs and more precise information concerning Stieltjes integrals, Laplace transforms, and complete monotonicity. Dahlquist [87] gives more details about applications to numerical methods.

The sequence defined by

t n n = dβ(t ), n = 0, 1, 2, . . . ,

is called a moment sequence if β(t) is nondecreasing. We make the convention that t 0 =1 also for t = 0, since the continuity of f is required in the definition of the Stieltjes integral.

Consider the special example where β(0) = 0, β(t) = 1 if t > 0. This means a unit point mass at t = 0, and no more mass for t > 0. Then u 0 = 1, u n = 0 for n > 0. It is then conceivable that making a sequence minimal just means removing a point mass from the origin; thus minimality means requiring that β(t) is continuous at t = 0. (For a proof, see [373, Sec. 4.14].)

The following theorem combines parts of several theorems in the books by Widder. It is important that the functions called α(x) and β(t) in this theorem need not to be explicitly known for an individual series for applications of an error estimate or a convergence rate of

a method of convergence acceleration. Some criteria will be given below that can be used for simple proofs that a particular series is (or is not) c.m. or d.c.m.

Theorem 3.4.5.

1. The sequence {u n } ∞ 0 is c.m. if and only if it is a moment sequence; it is minimal if in addition β(t) is continuous at t = 0, i.e., if there is no point mass at the origin. It is

a d.c.m. if and only if (

3.4.25) holds for some β(t) of bounded variation.

2. The function u(s) is c.m. for s ≥ 0 if and only if it can be represented as a Laplace– Stieltjes transform,

u(s) =

e −sx dα(x), s ≥ 0,

with a nondecreasing and bounded function α(x). For the open interval s > 0 we have the same, except for the boundedness of α(x). For a d.c.m. the same is true with α(x) of bounded variation (not necessarily bounded as x → ∞). The integral representation provides an analytic continuation of u(s) from a real interval to a half-plane.

3. The sequence {u n } ∞ 0 is a minimal c.m. if and only if there exists a c.m. function u(s) such that u n = u(n), n = 0, 1, 2, . . . .

3.4. Acceleration of Convergence 287

4. Suppose that u(s) is c.m. in the interval s > a. Then the Laplace–Stieltjes integral converges absolutely and uniformly if ℜs ≥ a ′ , for any a ′ > a, and defines an analytic

continuation of u(s) that is bounded for ℜs ≥ a ′ and analytic for ℜs > a. This is true also if u(s) is a d.c.m.

Proof. The “only if” parts of these statements are deep results mainly due to Hausdorff 97 and Bernštein, 98 and we omit the rather technical proofs. The relatively simple proofs of the “if” parts of the first three statements will be sketched, since they provide some useful insight.

1. Assume that u n is a moment sequence, β(0) = 0, β is continuous at t = 0 and non- decreasing for t > 0. Note that multiplication by E or 4 outside the integral sign in (3.4.25) corresponds to multiplication by t or t−1 inside. Then, for j, n = 0, 1, 2, . . . ,

− 1) j t n dβ(t )

j = n ( 1 − t) t dβ(t ) ≥ 0,

and hence u n is c.m.

2. Assume that u(s) satisfies (3.4.26). It is rather easy to legitimate the differentiation under the integral sign in this equation. Differentiation j times with respect to s yields, for j = 1, 2, 3, . . . ,

j −1) j u (j ) (s) j ∞

( −x) e −sx dα(x) =

x j e −sx dα(x) ≥ 0;

and hence u(s) is c.m.

0 e −nx dα(x) . Define t = e −x , β(0) = 0, β(t) ≡ β(e −x ) = u(0) − α(x), and note that

3. Assume that u n = u(n) = ∞

t = 0 ⇔ x = ∞, and that u(0) = lim x →∞ α(x) . It follows that β(t) is nonnegative and nondecreasing,

t = 1 ⇔ x = 0,

since x decreases as t increases. Note that β(t) ↓ β(0) as t ↓ 0. Then

n =−

t dβ(t ) =

t dβ(t ),

hence {u n } is a minimal c.m.

4. The distinction is illustrated for α ′ (x) =e ax , u(s) = (s − a) −1 , for a real a. u(s) is analytic for ℜs > a and bounded only for ℜs ≥ a ′ for any a ′ >a .

97 Felix Hausdorff (1868–1942), a German mathematician, is mainly known for having created a modern theory of topological and metric spaces.

98 Sergei Natanoviˇc Bernštein (1880–1968), Russian mathematician. Like his countryman Chebyshev, he made major contributions to polynomial approximation.

288 Chapter 3. Series, Operators, and Continued Fractions The basic formula for the application of complete monotonicity to the summation of

power series reads

i 1 i S(z) ≡

z t dβ(t ) =

z t dβ(t ) =

( 1 − zt) −1 dβ(t ).

i =0 0 0 0 0 0

(3.4.27) The inversion of the summation and integration is legitimate when |z| < 1. Note that the

last integral exists for more general z; a classical principle of complex analysis then yields the following interesting result.

Lemma 3.4.6.

If the sequence {u i } is d.c.m., then the last integral of formula (3.4.27) provides the unique single-valued analytic continuation of S(z) to the whole complex plane, save for a

cut along the real axis from 1 to ∞. Remark 3.4.3. When z is located in the cut, (1 − zt) −1 has a nonintegrable singularity

at t = 1/z ∈ [0, 1] unless, e.g., β(t) is constant in the neighborhood of this point. If we remove the cut, S(z) will not be single-valued. Check that this makes sense for β(t) = t.

Next we shall apply the above results to find interesting properties of the (generalized) Euler transformation. For example, we shall see that, for any z outside the cut, there is an optimal strategy for the generalized Euler transformation that provides the unique value of the analytic continuation of S(z). The classical Euler transformation, however, reaches only the half-plane ℜz < 1

2 . After that we shall see that there are a number of simple criteria for finding out

whether a given sequence is c.m., d.c.m., or neither. Many interesting sequences are c.m., for example, u n =e −kn ,u n = (n + c) −k , (k ≥ 0, c ≥ 0), all products of these, and all linear combinations (i.e., sums or integrals) of such sequences with positive coefficients.

The convergence of a c.m. toward zero can be arbitrarily slow, but an alternating series with c.m. terms will, after Euler’s transformation, converge as rapidly as a geometric series. More precisely, the following result on the optimal use of a generalized Euler transformation will be shown.

Theorem 3.4.7.

3.4.1 and (3.4.22). Suppose that the sequence {u j } is either c.m. or d.c.m. Consider

We use the notation of Theorem

S(z) =

u j j z , z ∈ C,

j =0

and its analytic continuation (according to the above lemma). Then for the classical Euler transformation the following holds: If z = −1, a sequence along a descending diagonal of

the scheme M or (equivalently) the matrix ¯ M, i.e., {M n 0 ,k } ∞ k =0 for a fixed n 0 , converges at least as fast as 2 −k . More generally, the error behaves like (z/(

1 − z)) k , (k ≫ 1). Note that |z/(1 − z)| < 1 if and only if ℜz < 1

2 . The classical Euler transformation diverges outside this half-plane. If z = e ±it , π

3 <t ≤ π, it converges as fast as (2 sin 2 ) −k .

3.4. Acceleration of Convergence 289 For the generalized Euler transformation we have the following: If z = −1, the

smallest error in the ith row of ¯ M is O( 3 −i ), as i → ∞. More generally, this error is O((

|z|/(1 + |1 − z|)) i ), hence the smallest error converges exponentially, unless z − 1 is real and positive; i.e., the optimal application of the generalized Euler’s transformation

provides the analytic continuation, whenever it exists according to Lemma

3.4.6. If N ≫ 1, the optimal value 99

of k/N is |1 − z|/(1 + |1 − z|). If z = e ±it , 0 < t ≤ π, the error is O(( 1 + 2 sin 2 ) −i ).

Proof. Sketch: The result of the generalized Euler transformation is in Sec. 3.4.3, denoted by M n,k (z) . The computation uses N = n + k terms (or partial sums) of the power series for S(z); n terms of the original series—the head—are added, and Euler’s transformation is applied to the next k terms—the tail. Set n/N = µ, i.e., n = µN, k = (1 − µ)N, and denote the error of M n,k by R N,µ (z) . Euler’s transformation is based on the operator P

= P (z) = z 1−z 4 z . A multiplication by the operator P corresponds to a multiplication by 1−z (t − 1) inside the integral sign. First suppose that |z| < 1. By the definitions of S(z) and M n,k (z) in Theorem 3.4.1,

N,µ (z) ≡S−M n,k =

1 − z) z n

P u n = t 1−z dβ(t )

t n dβ(t )

1−z 0 ( 1 − z)

1 − z(t − 1)/(1 − z)

dβ(t )

We see that the error oscillates as stated in Sec. 3.4.3. Again, by analytic continuation, this holds for all z except for the real interval [1, ∞]. Then

|dβ(t)| |R N,µ (z) 1/N

| 1/N ≤ |z/(1 − z) 1−µ | max ( 1 − t) 1−µ t c , c .

0 |1 − zt| The first part of the theorem has n = 0, hence µ = 0. We obtain

lim

N →∞ |R N, 0 | 1/N ≤ |z/(1 − z)|

as stated. This is less than unity if |z| < |1 − z|, i.e., if ℜ(z) < 1

2 . Now we consider the second part of the theorem. The maximum occurring in the

above expression for |R N,µ (z) 1/N | (with N, µ fixed) takes place at t = µ. Hence N,µ (z) |R 1/N | ≤ |z/(1 − z) 1−µ 1/N |c (

1−µ 1 − µ) µ µ . An elementary optimization shows that the value of µ that minimizes this bound for

|R 1/N N,µ (z) | is µ = 1/(|1 − z| + 1), i.e.,

99 In practice this is found approximately by the termination criterion of Algorithm 3.4.

290 Chapter 3. Series, Operators, and Continued Fractions and the minimum equals |z|/(|1 − z| + 1). The details of these two optimizations are left

for Problem 3.4.34. This proves the second part of the theorem. This minimum turns out to be a rather realistic estimate of the convergence ratio of the

optimal generalized Euler transformation for power series with d.c.m. coefficients, unless β(t ) is practically constant in some interval around t = µ; the exception happens, e.g., if u n

=a n Here we shall list a few criteria for higher monotonicity, by which one can often answer

the question of whether a function is c.m. or d.c.m. or neither. When several c.m. or d.c.m. are involved, the intervals should be reduced to the intersection of the intervals involved. By Theorem 3.4.5, the question is then also settled for the corresponding sequence. In simple cases the question can be answered directly by means of the definition or the above theorem, e.g., for u(s) = e −ks ,s −k , (k ≥ 0), for ℜs ≥ 0 in the first case, for ℜs > 0 in the second case.

(A) If u(s) is c.m., and a, b ≥ 0, then g(s) = u(as + b) and (−1) j u (j ) (s) are c.m., j

u(t ) dt is also c.m., if it is convergent. (The interval of complete monotonicity may not be the same for g as for f .) Analogous statements hold for sequences.

= 1, 2, 3, . . . . The integral ∞

(B) The product of two c.m. is c.m. Similarly, the product of two d.c.m. is d.c.m. This can evidently be extended to products of any number of factors, and hence to every positive integral power of a c.m. or d.c.m. The proof is left for Problem 3.4.34.

(C) A uniformly convergent positive linear combination of c.m. is itself c.m. The same criterion holds for d.c.m. without the requirement of positivity . The term “positive linear combination” includes sums with positive coefficients and, more generally, Stieltjes integrals u(s ; p) dγ (p), where γ (p) is nondecreasing.

(D) Suppose that u(s) is a d.c.m. for s ≥ a. F (u(s)) is then a d.c.m. for s > a, if the radius of convergence of the Taylor expansion for F (z) is greater than max |u(s)|.

Suppose that u(s) is c.m. for s ≥ a. We must then add the assumption that the coefficients of the Taylor expansion of F (z) are nonnegative, in order to make sure

that F (u(s)) is c.m. for s ≥ a. These statements are important particular cases of (C). We also used (B), according

to which each term u(s) k is c.m. (or a d.c.m. in the first statement). Two illustrations: g(s) = (1−e −s ) −1

is c.m. for s > 0; h(s) = (s 2 +1) −1 is a d.c.m. at least for s > 1 (choose z =s −2 ). The expansion into powers of s −2 also provides an explicit decomposition,

2 4 +s 4 −8 + · · ·) = s /(s − 1) − 1/(s − 1), where the two components are c.m. for s > 1. See also Example 3.4.8.

h(s) = (s −2 +s −6 + · · ·) − (s −4

(E) If g ′ (s) is c.m. for s > a, and if u(z) is c.m. in the range of g(s) for s > a, then

F (s) = u(g(s)) is c.m. for s > a. (Note that g(s) itself is not c.m.) For example, we shall show that 1/ ln s is c.m. for s > 1. Set g(s) = ln s, u(z) = z −1 ,

a = 1. Then u(z) is completely monotonic for z > 0, and g ′ (s) =s −1 is c.m. for s>

0, a fortiori for s > 1 where ln s > 0. Then the result follows from (E).

3.4. Acceleration of Convergence 291 The problems of Sec. 3.4 contain many interesting examples that can be treated by

means of these criteria. One of the most important is that every rational function that is analytic and bounded in a half-plane is d.c.m. there; see Problem 3.4.35. Sometimes a table of Laplace transforms (see, e.g., the Handbook [1, Chap. 29]) can be useful in combination with the criteria below.

Another set of criteria is related to the analytic properties of c.m. and d.c.m. functions. Let u(s) be d.c.m. for s > a. According to statement 4 of Theorem 3.4.5, u(s) is analytic and bounded for s ≥ a ′ for any a ′ >a . The converse of this is not unconditionally true. If, however, we add the conditions that

|u(σ + iω)| dω < ∞, u(s) → 0, as |s| → ∞, σ ≥a ′ , (3.4.29)

then it can be shown that u(s) is a d.c.m. for s > a. This condition is rather restrictive; there are many d.c.m. that do not satisfy it, for example, functions of the form e −ks or k + b(s − c) −γ (k ≥ 0, b ≥ 0, c > a, 0 < γ ≤ 1). The following is a reasonably powerful criterion: u(s) is a d.c.m. for s > a, e.g., if we can make a decomposition of the form

u(s) =f 1 (s) +f 2 (s) or u(s) =f 1 (s)f 2 (s), where f 1 (s) is known to be d.c.m. for s > a, and f 2 (s) satisfies the conditions in (3.4.29).

Theorem 3.4.8.

Suppose that u(s) is c.m. for some s though not for all s. Then a singularity on the real axis, at (say) s = a, must be among the rightmost singularities; u(s) is c.m. for s > a,

hence analytic for ℜs > a. The statement in the theorem is not generally true if u(s) is only d.c.m. Suppose that

u(s) is d.c.m. for s > a, though not for any s < a. Then we cannot even be sure that there exists a singularity s ∗ such that ℜs ∗ = a.

Example 3.4.8.

This theorem can be used for establishing that a given function is not a c.m. For example, u(s) = 1/(1 + s 2 ) is not c.m. since the rightmost singularities are s = ±i, while

s = 0 is no singularity. u(s) is a d.c.m. for s > 0; however, since it is analytic and bounded, and satisfies (3.4.29) for any positive a ′ . This result also comes from the general statement about rational functions bounded in a half-plane; see Problem 3.4.35.

Another approach: in any text about Laplace transforms you find that, for s > 0,

Now α ′ (x) ≥ 0 in both terms. Hence the formula (1/s + 1/(s 2 + 1)) − 1/s expresses 1/(s 2 + 1) as the difference of two c.m. sequences for s > 0.

The easy application of criterion (D) above gave a smaller interval (s > 1), but a faster decrease of the c.m. terms as s → ∞.

Another useful criterion for this kind of negative conclusion is that a c.m. sequence cannot decrease faster than every exponential as s → +∞, for s ∈ R, unless it is identically

292 Chapter 3. Series, Operators, and Continued Fractions zero. For there exists a number ξ such that α(ξ) > 0, hence

u(s) =

e −sx dα(x) ≥

e −sx dα(x) ≥e −sξ α(ξ ).

For example, e −s 2 and 1/ Ŵ(s) are not c.m. Why does this not contradict the fact that s −1 e −s is c.m.?

These ideas can be generalized. Suppose that {c i } ∞ i is a given sequence such that

the sum C(t) ≡ =0

i =0 c i t i is known, and that u i is c.m. or d.c.m. (c i and C(t) may depend on a complex parameter z too). Then

c t c i = i i = i dβ(t ) C(t ) dβ(t ).

i =0

i =0

It is natural to ask how well S c is determined if u i has been computed for i < N, if {u n } ∞ 0 is constrained to be c.m. A systematic way to obtain very good bounds is to find a polynomial Q ∈ P N such that |C(t) − Q(t)| ≤ ǫ N for all t ∈ [0, 1]. Then

|S c − Q(E)u 0 |= ( (

C(t ) − Q(t) dβ(t ) (

0 (≤ǫ 0 Note that Q(E)u 0 is a linear combination of the computed values u i ,i<N , with coeffi-

|dβ(t)|.

cients independent of {u n }. For C(t; z) = (1 − tz) −1 the generalized Euler transformation (implicitly) works with a particular array of polynomial approximations, based on Taylor expansion, first at t = 0 and then at t = 1.

Can we find better polynomial approximations? For C(t; z) = (1−tz) −1 , Gustafson’s Chebyshev acceleration (GCA) [177] is in most respects, superior to Euler transformation.

Like Euler’s transformation this is based on linear transformations of sequences and has the same range of application as the optimal Euler transformation. For GCA

ǫ 1/N

if z = −1. The number of terms needed for achieving a certain accuracy is thus for GCA √ about ln(3 + 8)/ ln 3 ≈ 1.6 times as large as for the optimal Euler transformation.