It may seem that Theorem 1 would automatically also ensure that S
n
→ ∞ also path-wise. This is not, however, the case. For example, consider the probability space [0, 1] with the Borel
σ-algebra and the Lebesgue measure. Then M
n
, F
n n
≥1
defined as M
n
:= 2
2n
1
[0,2
−n
and F
n
:= σX
k
: 1 ≤
k ≤ n is, in fact, a submartingale. Moreover, EM
n
= 2
n
→ ∞, but M
n
→ 0 almost surely. The AM process, however, does produce an unbounded sequence S
n
.
Theorem 5. Assume that X
n n
≥2
follows the ‘adaptive random walk’ recursion 6. Then, for any unit vector u
∈ R
d
, the process u
T
S
n
u → ∞ almost surely.
Proof. Theorem 5 is a special case of Theorem 18 in Section 3.3. In a one-dimensional setting, and when log
π is uniformly continuous, the AM process can be ap- proximated with the ‘adaptive random walk’ above, whenever S
n
is small enough. This yields
Theorem 6. Assume d = 1 and log π is uniformly continuous. Then, there is a constant b 0 such
that lim inf
n →∞
S
n
≥ b. Proof. Theorem 6 is a special case of Theorem 18 in Section 3.4.
Finally, having Theorem 6, it is possible to establish
Theorem 7. Assume ˜ q is Gaussian, the one-dimensional target distribution is standard Laplace
πx :=
1 2
e
−|x|
and the functional f : R → R satisfies sup
x
e
−γ|x|
| f x| ∞ for some γ ∈ 0, 12. Then, n
−1
P
n k=1
f X
k
→ R
f x πxdx almost surely as n → ∞.
Proof. Theorem 7 is a special case of Theorem 21 in Section 3.4. Remark 8. In the case
η
n
:= n
−1
, Theorem 7 implies that the parameters M
n
and S
n
of the adap- tive chain converge to 0 and 2, that is, the true mean and variance of the target distribution
π, respectively.
Remark 9. Theorem 6 and Theorem 7 could probably be extended to cover also targets π with
compact supports. Such an extension would, however, require specific handling of the boundary effects, which can lead to technicalities.
3.2 Uniform target: expected growth rate
Define the following matrix quantities a
n
:= E
X
n
− M
n −1
X
n
− M
n −1
T
7
b
n
:= E S
n
8 for n
≥ 1, with the convention that a
1
≡ 0 ∈ R
d ×d
. One may write using 3 and 6 X
n+1
− M
n
= X
n
− M
n
+ θ S
1 2
n
W
n+1
= 1 − η
n
X
n
− M
n −1
+ θ S
1 2
n
W
n+1
. If EW
n
W
T n
= I, one may easily compute E
X
n+1
− M
n
X
n+1
− M
n T
= 1 − η
n 2
E
X
n
− M
n −1
X
n
− M
n −1
T
+ θ
2
E S
n
51
since W
n+1
is independent of F
n
and zero-mean due to the symmetry of ˜ q. The values of a
n n
≥2
and b
n n
≥2
are therefore determined by the joint recursion a
n+1
= 1 − η
n 2
a
n
+ θ
2
b
n
9 b
n+1
= 1 − η
n+1
b
n
+ η
n+1
a
n+1
. 10
Observe that for any constant unit vector u ∈ R
d
, the recursions 9 and 10 hold also for a
u n+1
:= E
u
T
X
n+1
− M
n
X
n+1
− M
n T
u
b
u n+1
:= E
u
T
S
n+1
u
. The rest of this section therefore dedicates to the analysis if the one-dimensional recursions 9 and
10, that is, a
n
, b
n
∈ R
+
for all n ≥ 1. The first result shows that the tail of b
n n
≥1
is increasing.
Lemma 10. Let n
≥ 1 and suppose a
n
≥ 0, b
n
0 and for n ≥ n the sequences a
n
and b
n
follow the recursions 9 and 10, respectively. Then, there is a m
≥ n such that b
n n
≥m
is strictly increasing. Proof. If
θ ≥ 1, we may estimate a
n+1
≥ 1 − η
n 2
a
n
+ b
n
implying b
n+1
≥ b
n
+ η
n+1
1 − η
n 2
a
n
for all n ≥ n
. Since b
n
0 by construction, and therefore also a
n+1
≥ θ
2
b
n
0, we have that b
n+1
b
n
for all n ≥ n
+ 1. Suppose then
θ 1. Solving a
n+1
from 10 yields a
n+1
= η
−1 n+1
b
n+1
− b
n
+ b
n
Substituting this into 9, we obtain for n ≥ n
+ 1 η
−1 n+1
b
n+1
− b
n
+ b
n
= 1 − η
n 2
η
−1 n
b
n
− b
n −1
+ b
n −1
+ θ
2
b
n
After some algebraic manipulation, this is equivalent to b
n+1
− b
n
= η
n+1
η
n
1 − η
n 3
b
n
− b
n −1
+ η
n+1
1 − η
n 2
− 1 + θ
2
b
n
. 11
Now, since η
n
→ 0, we have that 1 − η
n 2
− 1 + θ
2
0 whenever n is greater than some n
1
. So, if we have for some n
′
n
1
that b
n
′
− b
n
′
−1
≥ 0, the sequence b
n n
≥n
′
is strictly increasing after n
′
. Suppose conversely that b
n+1
− b
n
0 for all n ≥ n
1
. From 10, b
n+1
− b
n
= η
n+1
a
n+1
− b
n
and hence b
n
a
n+1
for n ≥ n
1
. Consequently, from 9, a
n+1
1 − η
n 2
a
n
+ θ
2
a
n+1
, which is equivalent to
a
n+1
1 − η
n 2
1 − θ
2
a
n
. Since
η
n
→ 0, there is a µ 1 and n
2
such that a
n+1
≥ µa
n
for all n ≥ n
2
. That is, a
n n
≥n
2
grows at least geometrically, implying that after some time a
n+1
b
n
, which is a contradiction. To conclude, there is an m
≥ n such that b
n n
≥m
is strictly increasing. Lemma 10 shows that the expectation E
u
T
S
n
u
is ultimately bounded from below, assuming only that
η
n
→ 0. By additional assumptions on the sequence η
n
, the growth rate can be characterised in terms of the adaptation weight sequence.
52
Assumption 11. Suppose η
n n
≥1
⊂ 0, 1 and there is m
′
≥ 2 such that i
η
n n
≥m
′
is decreasing with η
n
→ 0, ii
η
−12 n+1
− η
−12 n
n ≥m
′
is decreasing and iii
P
∞ n=2
η
n
= ∞. The canonical example of a sequence satisfying Assumption 11 is the one assumed in Section 3.1,
η
n
:= cn
−γ
for c ∈ 0, 1 and γ ∈ 12, 1].
Theorem 12. Suppose a
m
≥ 0 and b
m
0 for some m ≥ 1, and for n m the a
n
and b
n
are given recursively by 9 and 10, respectively. Suppose also that the sequence
η
n n
≥2
satisfies Assumption 11 with some m
′
≥ m. Then, for all λ 1 there is m
2
≥ m
′
such that for all n ≥ m
2
and k ≥ 1, the
following bounds hold 1
λ
θ
n+k
X
j=n+1
pη
j
≤ log
b
n+k
b
n
≤ λ
θ
n+k
X
j=n+1
pη
j
.
Proof. Let m be the index from Lemma 10 after which the sequence b
n
is increasing. Let m
1
max {m
, m
′
} and define the sequence z
n n
≥m
1
−1
by setting z
m
1
−1
= b
m
1
−1
and z
m
1
= b
m
1
, and for n
≥ m
1
through the recursion z
n+1
= z
n
+ η
n+1
η
n
1 − η
n 3
z
n
− z
n −1
+ η
n+1
˜ θ
2
z
n
12 where ˜
θ 0 is a constant. Consider such a sequence z
n n
≥m
1
−1
and define another sequence g
n n
≥m
1
+1
through g
n+1
:= η
−12 n+1
z
n+1
− z
n
z
n
= η
−12 n+1
η
n+1
η
n
1 − η
n 3
z
n
− z
n −1
z
n −1
z
n −1
z
n
+ η
n+1
˜ θ
2
= η
1 2
n+1
1 − η
n 3
η
n
g
n
g
n
+ η
−12 n
+ ˜ θ
2
. Lemma 33 in Appendix A shows that g
n
→ ˜ θ .
Let us consider next two sequences z
1 n
n ≥m
1
−1
and z
2 n
n ≥m
1
−1
defined as z
n n
≥m
1
−1
above but using two different values ˜
θ
1
and ˜ θ
2
, respectively. It is clear from 11 that for the choice ˜
θ
1
:= θ one has b
n
≤ z
1 n
for all n ≥ m
1
− 1. Moreover, since b
m
1
+1
b
m
1
≤ z
1 m
1
+1
z
1 m
1
, it holds by induction that
b
n+1
b
n
≤ 1 + η
n+1
η
n
1 − η
n 3
1 −
b
n −1
b
n
+ η
n+1
˜ θ
2
≤ 1 + η
n+1
η
n
1 − η
n 3
1 −
z
1 n
−1
z
1 n
+ η
n+1
˜ θ
2
= z
1 n+1
z
1 n
also for all n ≥ m
1
+ 1. By a similar argument one shows that if ˜ θ
2
:= [1 − η
m
1
2
− 1 + θ
2
]
1 2
then b
n
≥ z
2 n
and b
n+1
b
n
≥ z
2 n+1
z
2 n
for all n ≥ m
1
− 1. 53
Let λ
′
1. Since g
1 n
→ ˜ θ
1
and g
2 n
→ ˜ θ
2
there is a m
2
≥ m
1
such that the following bounds apply
1 + ˜
θ
2
λ
′
p η
n
≤ z
2 n
z
2 n
−1
and z
1 n
z
1 n
−1
≤ 1 + λ
′
˜ θ
1
p η
n
for all n ≥ m
2
. Consequently, for all n ≥ m
2
, we have that log
b
n+k
b
n
≤ log
z
1 n+k
z
1 n
≤
n+k
X
j=n+1
log 1 +
λ
′
˜ θ
1
pη
j
≤ λ
′
θ
n+k
X
j=n+1
p η
n
. Similarly, by the mean value theorem
log b
n+k
b
n
≥
n+k
X
j=n+1
log
1 + ˜
θ
2
λ
′
pη
j
≥
˜ θ
2
λ
′
1 + λ
′−1
˜ θ
2
p η
n n+k
X
j=n+1
pη
j
since η
n
is decreasing. By letting the constant m
1
above be sufficiently large, the difference | ˜
θ
2
−θ | can be made arbitrarily small, and by increasing m
2
, the constant λ
′
1 can be chosen arbitrarily close to one.
3.3 Uniform target: path-wise behaviour
