getdocc094. 145KB Jun 04 2011 12:04:55 AM
Elect. Comm. in Probab. 13 (2008), 614–627
ELECTRONIC
COMMUNICATIONS
in PROBABILITY
MARKOV PROCESSES WITH PRODUCT-FORM STATIONARY DISTRIBUTION
KRZYSZTOF BURDZY1
Department of Mathematics, University of Washington, Seattle, WA 98195
email: burdzy@math.washington.edu
DAVID WHITE
Department of Mathematics, Belmont University, Nashville, TN 37212
email: white@u.washington.edu
Submitted November 3, 2007, accepted in final form November 18, 2008
AMS 2000 Subject classification: 60J25
Keywords: Markov process, stationary distribution, inert drift
Abstract
We consider a continuous time Markov process (X , L), where X jumps between a finite number of
states and L is a piecewise linear process with state space Rd . The process L represents an “inert
drift” or “reinforcement.” We find sufficient and necessary conditions for the process (X , L) to have
a stationary distribution of the product form, such that the marginal distribution of L is Gaussian.
We present a number of conjectures for processes with a similar structure but with continuous
state spaces.
1 Introduction
This research has been inspired by several papers on processes with inert drift [5, 4, 3, 1]. The
model involves a “particle” X and an “inert drift” L, neither of which is a Markov process by
itself, but the vector process (X , L) is Markov. It turns out that for some processes (X , L), the
stationary measure has the product form; see [1]. The first goal of this note is to give an explicit
characterization of all processes (X , L) with a finite state space for X and a product form stationary
distribution—see Theorem 2.1.
The second, more philosophical, goal of this paper is to develop a simple tool that could help
generate conjectures about stationary distributions for processes with continuous state space and
inert drift. So far, the only paper containing a rigorous result about the stationary distribution for
a process with continuous state space and inert drift, [1], was inspired by computer simulations.
Examples presented in Section 3 lead to a variety of conjectures that would be hard to arrive at
using pure intuition or computer simulations.
1
PARTIALLY SUPPORTED BY NSF GRANT DMS-0600206
614
Product-form stationary distribution
615
2 The model
Let S = {1, 2, . . . , N } for some integer N > 1 and let d ≥ 1 be an integer. We define a continuous
time Markov process (X (t), L(t)) on S × Rd as follows. We associate with each state j ∈ S a
vector v j ∈ Rd ,P
1 ≤ j ≤ N . Define L j (t) = µ({s ∈ [0, t] : X (s) = j}), where µ is Lebesgue measure,
and let L(t) = j∈S v j L j (t). To make the “reinforcement” non-trivial, we assume that at least one
of v j ’s is not 0. Since L will always belong to the hyperplane spanned by v j ’s, we also assume that
d = dim(span{v1 , . . . , vN }).
We also select non-negative functions ai j (l) which define the Poisson rates of jumps from state i
to j. The rates depend on l = L(t). We assume that ai j ’s are right-continuous with left limits.
Formally speaking, the process (X , L) is defined by its generator A as follows,
X
a ji (l)[ f (i, l) − f ( j, l)], j = 1, . . . , N , l ∈ Rd ,
Af ( j, l) = v j · ∇l f ( j, l) +
i6= j
for f : {1, . . . , N } × Rd → R.
We assume that (X , L) is irreducible in the sense of Harris, i.e., for some open set U ⊂ Rd and
some j0 ∈ S , for all (x, l) ∈ S × Rd , we have for some t > 0,
P((X (t), L(t)) ∈ { j0 } × U) > 0.
We are interested only in processes satisfying (14) below. Using that condition, it is easy to check
Harris irreducibility for each of our models by a direct argument. A standard coupling argument
shows that Harris irreducibility implies uniqueness of the stationary probability distribution (assuming existence of such).
The (formal) adjoint of A is given by
X
A∗ g( j, l) = −v j · ∇l g( j, l) +
[ai j (l)g(i, l) − a ji (l)g( j, l)], j = 1, . . . , N , l ∈ Rd .
(1)
i6= j
We are interested in invariant measures of product form so suppose that g( j, l) = p j g(l), where
R
P
j∈S p j = 1 and Rd g(l)d l = 1. We may assume that p j > 0 for all j; otherwise some points in
S are never visited. Under these assumptions, (1) becomes
X
[pi ai j (l)g(l) − p j a ji (l)g(l)], j = 1, . . . , N , l ∈ Rd .
A∗ g( j, l) = −p j v j · ∇g(l) +
i6= j
Theorem 2.1. Assume that for every i and j, the function l → ai j (l) is continuous. A probability
measure p j g(l)d jd l is invariant for the process (X , L) if and only if
X
− p j v j · ∇g(l) +
[pi ai j (l)g(l) − p j a ji (l)g(l)] = 0, j = 1, . . . , N , l ∈ Rd .
(2)
i6= j
Proof. Recall that the state space S for X is finite. Hence v∗ := sup j∈S |v j | < ∞. Fix arbitrary
r, t ∗ ∈ (0, ∞). It follows that,
sup
ai j (l) = a∗ < ∞.
i, j∈S ,l∈B(0,r+2t ∗ v∗ )
Note that we always have |L(t) − L(u)| ≤ v∗ |t − u|. Hence, if |L(0)| ≤ r + t ∗ v∗ and s, t > 0,
s + t ≤ t ∗ , then |L(s + t)| ≤ r + 2t ∗ v∗ and, therefore,
sup
j∈S ,u≤s+t
aX (u), j (L(u)) ≤ a∗ < ∞.
616
Electronic Communications in Probability
This implies that the probability of two or more jumps on the interval [s, s + t] is o(t). Assume
that |l| ≤ r + t ∗ v∗ and t ≤ t ∗ . Then we have the following three estimates. First,
P(X (t) = j | X (0) = i, L(0) = l) = ai j (l)t + R1i, j,l (t),
(3)
where the remainder R1i, j,l (t) satisfies supi, j∈S ,l∈B(0,t ∗ v∗ ) |R1i, j,l (t)| ≤ R1 (t) for some R1 (t) such that
lim t→0 R1 (t)/tP
= 0.
Let aii (l) = − j6=i ai j (l). We have
P(X (t) = i, L(t) = l + t vi | X (0) = i, L(0) = l) = 1 + aii (l)t + R2i,l (t),
(4)
where the remainder R2i,l (t) satisfies supi∈S ,l∈B(0,t ∗ v∗ ) |R2i,l (t)| ≤ R2 (t) for some R2 (t) such that
lim t→0 R2 (t)/t = 0.
Finally,
P(X (t) = i, L(t) 6= l + t vi | X (0) = i, L(0) = l) = R3i,l (t),
(5)
where the remainder R3i,l (t) satisfies supi∈S ,l∈B(0,t ∗ v∗ ) |R3i,l (t)| ≤ R3 (t) for some R3 (t) such that
lim t→0 R3 (t)/t = 0.
Now consider any C 1 function f ( j, l) with support in S × B(0, r). Recall that |L(t) − L(u)| ≤
v∗ |t − u|. Hence, Ei,l f (X t , L t ) = 0 for t ≤ t ∗ and |l| ≥ r + v∗ t ∗ .
Suppose that |l0 | ≤ r + v∗ t ∗ , t 1 ∈ (0, t ∗ ) and s ∈ (0, t ∗ − t 1 ). Then
Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l f (X t 1 , L t 1 )
Z
Z
X
X
=
f (k, r)P(X (t 1 + s) = k, L(t 1 + s) ∈ d r | X (t 1 ) = j, L(t 1 ) = l)
j∈S
Rd k∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
−
f ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d r | X (0) = i, L(0) = l0 ).
j∈S
Rd
We combine this formula with (3)-(5) to see that,
Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l f (X t 1 , L t 1 )
Z
X
X
=
( f (k, l) + O(s))(a jk (l)s + R1j,k,l (s))
j∈S
Rd k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
+
f ( j, l + sv j )(1 + a j j (l)s + R2j,l (s))
j∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
+
( f ( j, l) + O(s))R3j,l (s)
j∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
−
f ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ),
j∈S
Rd
Product-form stationary distribution
617
which can be rewritten as
X
Z
( f ( j, l + sv j ) − f ( j, l))P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
j∈S
+
X
Z
f ( j, l + sv j )a j j (l) +
Rd
j∈S
X
f (k, l)a jk (l) s
k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
X
1
1
f (k, l)R j,k,l (s) + O(s)(a jk (l)s + R j,k,l (s))
+
Rd
j∈S
k∈S ,k6= j
+ f ( j, l
+ sv j )R2j,l (s) + ( f
( j, l) + O(s))R3j,l (s)
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ).
We will analyze the limit
1
lim (Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l0 f (X t 1 , L t 1 )).
s↓0 s
Note that
lim
s↓0
1X
s
Z
Rd
j∈S
+ f ( j, l
X
f
(k, l)R1j,k,l (s) + O(s)(a jk (l)s
k∈S ,k6= j
+ sv j )R2j,l (s) + ( f
+ R1j,k,l (s))
( j, l) + O(s))R3j,l (s)
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
= 0.
We also have
lim
s↓0
=
1X
s
X
Z
j∈S
( f ( j, l + sv j ) − f ( j, l))P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
Z
∇l f ( j, l) · v j P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ),
Rd
j∈S
and
lim
s↓0
1X
s
j∈S
Z
f ( j, l + sv j )a j j (l) +
Rd
X
f (k, l)a jk (l) s
k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
X
=
f (k, l)a jk (l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ).
j∈S
Rd k∈S
618
Electronic Communications in Probability
This implies that
d
dt
=
¯
¯
Ei,l0 f (X t , L t )¯
X
j∈S
=
X
j∈S
1
= lim (Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l0 f (X t 1 , L t 1 ))
s↓0 s
t=t 1
!
X
∇l f ( j, l) · v j +
f (k, l)a jk (l) P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ) (6)
Z
Rd
k∈S
Z
Af ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
= Ei,l0 Af (X t 1 , L t 1 ),
(7)
for all i and |l0 | ≤ r + v∗ t ∗ .
We will argue that
P((X (t 1 ), L(t 1 )) ∈ · | X (0) = i, L(0) = l) → P((X (t 1 ), L(t 1 )) ∈ · | X (0) = i, L(0) = l0 ),
weakly when l → l0 . Let Tk be the time of the k-th jump of X . We have
Z t
P(T1 > t | X (0) = i, L(0) = l) = exp
(8)
aii (l + svi )ds .
0
Since l → aii (l) is continuous, we conclude that
P(T1 > t | X (0) = i, L(0) = l) → P(T1 > t | X (0) = i, L(0) = l0 ),
weakly as l → l0 . This and continuity of l → ai j (l) for every j implies that
P((X (T1 ), L(T1 )) ∈ · | X (0) = i, L(0) = l) → P((X (T1 ), L(T1 )) ∈ · | X (0) = i, L(0) = l0 ),
weakly when l → l0 . By the strong Markov property applied at Tk ’s, we obtain inductively that
P((X (Tk ), L(Tk )) ∈ · | X (0) = i, L(0) = l) → P((X (Tk ), L(Tk )) ∈ · | X (0) = i, L(0) = l0 ),
when l → l0 , for every k ≥ 1. This easily implies (8), because the number of jumps is stochastically
bounded on any finite interval.
P
Since ( j, l) ,→ ∇l f ( j, l) · v j +¯ k∈S f (k, l)a jk (l) is a continuous function, it follows from (6) and
¯
is continuous on the set |l0 | ≤ r + v∗ t ∗ .
(8) that l0 ,→ ddt Ei,l0 f (X t , L t )¯
t=t 1
¯
¯
Recall that Ei,l0 f (X t , L t ) = 0 for t ≤ t ∗ and |l0 | ≥ r + v∗ t ∗ . Hence, l0 → ddt Ei,l0 f (X t , L t )¯
is
t=t 1
continuous for all t 1 ≤ t ∗ and all values of i.
Fix some t ≤ t ∗ and let u t ( j, l) = E j,l f (X t , L t ). We have just shown that for a fixed t ≤ t ∗ and any
j, the function l → u t ( j, l) is C 1 . Hence we can apply (7) with f ( j, l) = u t ( j, l) to obtain,
1
E j,l f (X t , L t ) = lim (E j,l f (X t+s , L t+s ) − E j,l f (X t , L t ))
s↓0 s
dt
1
= lim (E j,l u t (X s , Ls ) − u t ( j, l))
s↓0 s
= (Au t )( j, l).
d
(9)
Product-form stationary distribution
619
P
Since sup j,l ∇l f ( j, l) · v j + k∈S f (k, l)a jk (l) < ∞, formula (6) shows that
d
sup
E j,l f (X s , Ls ) < ∞.
dt
j,l,s≤t ∗
(10)
Now assume that (2) is true and let π(d j, d l) = p j g(l)d jd l. In view of (10), we can change the
order of integration in the following calculation. For 0 ≤ t 1 < t 2 ≤ t ∗ , using (9),
Eπ f (X (t 2 ), L(t 2 )) − Eπ f (X (t 1 ), L(t 1 ))
Z
Z
X
X
E j,l f (X t 2 , L t 2 )p j g(l)d l −
=
j∈S
=
X
j∈S
Z
t2
=
Z
t1
t2
=
t1
Rd
Z
j∈S
Z
Rd
X
j∈S
X
j∈S
t2
t1
Z
d
ds
d
Rd
ds
(11)
E j,l f (X t 1 , L t 1 )p j g(l)d l
Rd
E j,l f (X s , Ls )dsp j g(l)d l
E j,l f (X s , Ls )p j g(l)d l ds
Z
(Aus )( j, l)p j g(l)d l ds.
Rd
Let h( j, l) = p j g(l). For a fixed j and s ≤ t ∗ , the function us ( j, l) = 0 outside a compact set, so we
can use integration by parts to show that
Z
Z
X
X
us ( j, l)(A∗ h)( j, l)d l.
(12)
(Aus )( j, l)p j g(l)d l =
j∈S
Rd
j∈S
Rd
We combine this with the previous formula and the assumption that A∗ h ≡ 0 to see that
Z t2
Z
X
Eπ f (X (t 2 ), L(t 2 )) − Eπ f (X (t 1 ), L(t 1 )) =
us ( j, l)(A∗ h)( j, l)d l ds = 0.
t1
j∈S
Rd
It follows that t → Eπ f (X (t), L(t)) is constant for every C 1 function f ( j, l) with compact support.
This proves that the distributions of (X (t 1 ), L(t 1 )) and (X (t 2 ), L(t 2 )) are identical under π, for all
0 ≤ t1 < t2 ≤ t∗.
Conversely, assume that π(d j, d l) = p j g(l)d jd l is invariant. Then the left hand side of (11) is
zero for all 0 ≤ t 1 < t 2 ≤ t ∗ . This implies that
Z
X
(Aus )( j, l)p j g(l)d l = 0
j∈S
Rd
for a set of s that is dense on [0, ∞). By (12),
Z
X
us ( j, l)(A∗ h)( j, l)d l = 0
j∈S
(13)
Rd
for a set of s that is dense on [0, ∞). Note that lims↓0 us ( j, l) = f ( j, l). Hence, the collection of
C 1 functions us ( j, l), obtained by taking arbitrary C 1 functions f ( j, l) with compact support and
positive reals s dense in [0, ∞), is dense in the family of C 1 functions with compact support. This
and (13) imply that A∗ h ≡ 0, that is, (2) holds.
620
Electronic Communications in Probability
Corollary 2.2. If a probability measure p j g(l)d jd l is invariant for the process (X , L) then
X
p j v j = 0.
(14)
j∈S
Proof. Summing (2) over j, we obtain
X
−p j v j · ∇g(l) = 0,
(15)
j∈S
for all l. Since g is integrable over Rd , it is standard to show that there exist l1 , l2 , . . . , l d which
span Rd . Applying (15) to all l1 , l2 , . . . , we obtain (14).
It will be convenient to use the following notation,
bi j (l) = pi ai j (l) − p j a ji (l).
(16)
Note that bi j = −b ji .
Corollary 2.3. A probability measure p j g(l)d jd l is invariant for the process (X , L) and g(l) is the
Gaussian density
g(l) = (2π)−d/2 exp(−|l|2 /2),
if and only if the following equivalent conditions hold,
X
[pi ai j (l) − p j a ji (l)] = 0,
p j vj · l +
i6= j
p j vj · l +
X
bi j (l) = 0,
(17)
j = 1, . . . , N , l ∈ Rd ,
j = 1, . . . , N , l ∈ Rd .
(18)
(19)
i6= j
Proof. If g(l) is the Gaussian density then ∇g(l) = −l g(l) and (2) is equivalent to (18). Conversely, if (2) and (18) are satisfied then ∇g(l) = −l g(l), so g(l) must have the form (17).
In the rest of the paper we will consider only processes satisfying (18)-(19).
Example 2.4. We now present some choices for ai j ’s. Recall the notation x + = max(x, 0), x − =
− min(x, 0), and the fact that x + − x − = x. Given v j ’s, p j ’s and bi j ’s which satisfy (19) and the
condition bi j = −b ji , we may take
+
ai j (l) = bi j (l) /pi .
(20)
Then
so
+
+
−
a ji (l) = b ji (l) /p j = −bi j (l) /p j = bi j (l) /p j ,
+
−
pi ai j (l) − p j a ji (l) = bi j (l) − bi j (l) = bi j (l),
as desired.
The above is a special case, in a sense, of the following. Suppose that p j = pi for all i and j.
Assume that v j ’s and bi j ’s satisfy (19) and the condition bi j = −b ji . Fix some c > 0 and let
ai j (l) =
bi j (l) exp(c bi j (l))
exp(c bi j (l)) − exp(−c bi j (l))
.
(21)
It is elementary to check that with this definition, (16) is satisfied for all i and j, because bi j (l) =
−b ji (l). The formula (21) arose naturally in [5]. Note that (20) (with all pi ’s equal) is the limit
of (21) as c → ∞.
Product-form stationary distribution
621
3 Approximation of processes with continuous state
space
This section is contains examples of processes (X , L) with finite state space for X , and conjectures
concerned with processes with continuous state space. There are no proofs in this section
First we will consider processes that resemble diffusions with reflection. In these models, the
“inert drift” is accumulated only at the “boundary” of the domain.
We will now assume that elements of S are points in a Euclidean space Rn with n ≤ N . We denote
them S = {x 1 , x 2 , . . . , x N }. In other words, by abuse of notation, we switch from j to x j . We also
take v j ∈ Rn , i.e. d = n. Moreover, we limit ourselves to functions bi j (l) of the form bi j · l for some
vector bi j ∈ Rn . Then (19) becomes
0 + b12 + b13 + · · · + b1N = −p1 v1
−b12 − 0
+ b23 + · · · + b2n = −p2 v2
(22)
...
−b1N − b2N − b3N − · · · − 0
= −p N vN .
Consider any orthogonal transformation Λ : Rn → Rn . If {bi j , v j , p j } satisfy (22) then so do
{Λbi j , Λv j , p j }.
Suppose that ai j (l) have the form ai j · l for some ai j ∈ Rn . If {ai j , v j , p j } satisfy (18) then so do
{Λai j , Λv j , p j }. Moreover, the process with parameters {ai j , v j } has the same transition probabilities as the one with parameters {Λai j , Λv j }.
Example 3.1. Our first example is a reflected random walk on the interval [0, 1]. Let x j =
( j − 1)/(N − 1) for j = 1, . . . , N . We will construct a process with all p j ’s equal to each other, i.e.,
p j = 1/N . We will take l ∈ R1 , v1 = α and vN = −α, for some α = α(N ) > 0, and all other v j = 0,
so that the “inert drift” L changes only at the endpoints of the interval. We also allow jumps only
between adjacent points, so bi j = 0 for |i − j| > 1. Then (22) yields
b12 = −α/N
−b12 + b23 = 0
···
−b(N −1)N = α/N .
Solving this, we obtain bi(i+1) = −α/N for all i.
We would like to find a family of semi-discrete models indexed by N that would converge to a
continuous process with product-form stationary distribution as N → ∞. For 1 < i < N , we set
ai(i+1) (l) = AR (l, N ) and a(i+1)i (l) = A L (l, N ). We would like the random walk to have variance of
order 1 at time 1, for large N , so we need
AR + A L = N 2 .
(23)
Since bi(i+1) = −α/N for all i, AR and A L have to satisfy
A L − AR = αl.
(24)
When l is of order 1, we would like to have drift of order 1 at time 1, so we take α = N . Then
(24) becomes
A L − AR = N l.
(25)
622
Electronic Communications in Probability
Solving (23) and (25) gives
AL =
N2 + Nl
2
AR =
,
N2 − Nl
2
.
Unfortunately, A L and AR given by the above formula can take negative values—this is not allowed
because ai j ’s have to be positive. However, for every N , the stationary distribution of L is standard
normal, so l typically takes values of order 1. We are interested in large N so, intuitively speaking,
AR and A L are not likely to take negative values. To make this heuristics rigorous, we modify the
formulas for AR and A L as follows,
AL =
N2 + Nl
2
∨ 0 ∨ N l,
AR =
N2 − Nl
2
∨ 0 ∨ (−N l).
(26)
Let PN denote the distribution of (X , L) with the above parameters. We conjecture that as N → ∞,
PN converge to the distribution of reflected Brownian motion in [0, 1] with inert drift, as defined
in [5, 1]. The stationary distribution for this continuous time process is the product of the uniform
measure in [0, 1] and the standard normal; see [1].
Example 3.2. This example is a semi-discrete approximation to reflected Brownian motion in a
bounded Euclidean subdomain of Rn , with inert drift. In this example we proceed in the reversed
order, starting with bi j ’s and ai j ’s.
Consider an open bounded connected set D ⊂ Rn . Let K be a (large) integer and let DK = Zn /K∩D,
i.e., DK is the subset of the square lattice with mesh 1/K that is inside D. We assume that DK is
connected, i.e., any vertices in DK are connected by a path in DK consisting of edges of length
1/K. We take S = DK and l ∈ Rn .
We will consider nearest neighbor random walk, i.e., we will take ai j (l) = 0 for |x i − x j | > 1/K.
In analogy to (26), we define
ai j (l) =
K2
2
(1 + (x i − x j ) · l) ∨ 0 ∨ K 2 (x i − x j ) · l.
(27)
Then bi j (l) = (K 2 /N )(x i − x j ) · l. Let us call a point in S = DK an interior point if it has 2n
neighbors in DK . We now define v j ’s using (22) with p j = 1/|DK |. For all interior points x j , the
vector v j is 0, by symmetry. For all boundary (that is, non-interior) points x j , the vector v j is not
0.
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. We conjecture that as K → ∞, PK converge to the distribution of normally reflected
Brownian motion in D with inert drift, as defined in [5, 1]. If D is C 2 then it is known that the
stationary distribution for this continuous time process is the product of the uniform measure in
D and the standard Gaussian distribution; see [1].
The next two examples are discrete counterparts of processes with continuous state space and
smooth inert drift. The setting is similar to that in Example 3.2. We consider an open bounded
connected set D ⊂ Rn . Let K be a (large) integer and let DK = Zn /K ∩ D, i.e., DK is the subset
of the square lattice with mesh 1/K that is inside D. We assume that DK is connected, i.e., any
vertices in DK are connected by a path in DK consisting of edges of length 1/K. We take S = DK
and l ∈ Rn .
Product-form stationary distribution
623
Example 3.3. This example is concerned with a situation when the stationary distribution has the
form p j g(l) where p j ’s are not necessarily equal. We start with a C 2 “potential” V : D → R. We
will write Vj instead of V (x j ). Let p j = c exp(−Vj ). We need an auxiliary function
di j =
2(pi − p j )
pi (Vj − Vi ) − p j (Vi − Vj )
.
Note that di j = d ji and for a fixed i, we have di jK → 1 when K → ∞ and |i − jK | = 1/K.
Let ai j (l) = 0 for |x i − x j | > 1/K, and for |x i − x j | = 1/K,
ei j (l) =
a
We set
¨
ai j (l) =
K2
2
(2 + di j (Vi − Vj ) + (x j − x i ) · l).
ei j (l) ∨ 0
a
(K 2 /2pi )(pi + p j )(x j − x i ) · l
e ji (l) > 0,
if a
otherwise.
(28)
e ji (l) > 0 and a
ei j (l) > 0 then
If a
bi j (l) = pi ai j (l) − p j a ji (l)
=
=
=
K2
2
K2
2
K2
2
(2(pi − p j ) + (pi (Vi − Vj ) − p j (Vj − Vi ))di j + (pi (x j − x i ) − p j (x i − x j )) · l)
(2(pi − p j ) − 2(pi − p j ) + (pi + p j )(x j − x i ) · l)
(pi + p j )(x j − x i ) · l.
e ji (l) ≤ 0 or a
ei j (l) ≤ 0. Consider an
It follows from (28) that the above formula holds also if a
interior point x j . For (19) to be satisfied, we have to take
vj = −
X
1
pj
|x i −x j |=1/K
K2
2
(pi + p j )(x j − x i ).
For large K, series expansion shows that
v j ≈ −∇V.
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. We recall the following SDE from [1],
d Yt = −∇V (Yt ) d t + S t d t + d B t ,
dS t = −∇V (Yt ) d t ,
where B is standard n-dimensional Brownian motion and V is as above. Let P∗ denote the distribution of (Y, S). We conjecture that as K → ∞, PK converge to P∗ . Under mild assumptions on V ,
it is known that the stationary distribution for (Y, S) is the product of the measure exp(−V (x))d x
and the standard Gaussian distribution; see [1].
624
Electronic Communications in Probability
Example 3.4. We again consider the situation when all p j ’s are equal, i.e., p j = 1/N . Consider a
C 2 function V : D → R. We let ai j (l) = 0 for |x i − x j | > 1/K. If |x i − x j | = 1/K, we let
ei j (l) =
a
We set
¨
ai j (l) =
K2
2
(1 + (Vj + Vi )(x j − x i ) · l).
ei j (l) ∨ 0
a
K 2 (Vj + Vi )(x j − x i ) · l
e ji (l) > 0,
if a
otherwise.
(29)
Then bi j (l) = (1/N )K 2 (Vj + Vi )(x j − x i ) · l and
X
vj = K 2
(Vj + Vi )(x j − x i ).
|x i −x j |=1/K
For large K, we have v j ≈ −2∇V .
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. Consider the following SDE,
d Yt = V (Yt )S t d t + d B t ,
dS t = −2∇V (Yt ) d t ,
where B is standard n-dimensional Brownian motion and V is as above. Let P∗ denote the distribution of (Y, S). We conjecture that as K → ∞, PK converge to P∗ , and that the stationary distribution
for (Y, S) is the product of the uniform measure on D and the standard Gaussian distribution.
The next example and conjecture are devoted to examples where the inert drift is related to the
curvature of the state space, in a suitable sense.
Example 3.5. In this example, we will identify R2 and C. The imaginary unit will be denoted by i,
as usual. Let S consist of N points on a circle with radius r > 0, x j = r exp( j2πi/N ), j = 1, . . . , N .
We assume that the p j ’s are all equal to each other.
For any pair of adjacent points x j and x k , we let
e jk (l) =
a
and
¨
a jk (l) =
N2
2
(1 + (x k − x j ) · l),
e jk (l) ∨ 0
a
N 2 (x k − x j ) · l
ek j (l) > 0,
if a
otherwise,
with the other ak j (l) = 0. Then b j( j+1) = N (x j+1 − x j ) · l, and by (19) we have
v j = N 2 (x j−1 − x j ) + N 2 (x j+1 − x j ) = 2N 2 (cos(2π/N ) − 1)x j .
Note that v j → −4π2 x j when N → ∞.
Let PN be the distribution of (X , L) constructed above.
Let C be the circle with radius r > 0 and center 0, and let T y be the projection of R2 onto the
tangent line to C at y ∈ C . Consider the following SDE,
d Yt = TYt (S t ) d t + d B t ,
dS t = −4π2 Yt d t ,
Product-form stationary distribution
625
where Y takes values in C and B is Brownian motion on this circle. Let P∗ be the distribution of
(Y, S). We conjecture that as N → ∞, PN converge to P∗ , and that the stationary distribution for
(Y, S) is the product of the uniform measure on the circle and the standard Gaussian distribution.
Conjecture 3.6. We propose a generalization of the conjecture stated in the previous example.
We could start with an explicit discrete approximation, just like in other examples discussed so far.
The notation would be complicated and the whole procedure would not be illuminating, so we
skip the approximation and discuss only the continuous model.
Let S ⊂ Rn be a smooth (n − 1)-dimensional surface, let T y be the projection of Rn onto the
tangent space to S at y ∈ S , let n( y) be the inward normal to S at y ∈ S , and let ρ be the
mean curvature at y ∈ S . Consider the following SDE,
d Yt = TYt (S t ) d t + d B t ,
dS t = c0 ρ −1 n(Yt ) d t ,
where Y takes values in S and B is Brownian motion on this surface. We conjecture that for some
c0 depending only on the dimension n, the stationary distribution for (Y, S) exists, is unique and
is the product of the uniform measure on S and the standard Gaussian distribution.
We end with examples of processes that are discrete approximations of continuous-space processes
with jumps. It is not hard to construct examples of discrete-space processes that converge in
distribution to continuous-space processes with jumps. Stable processes are a popular family of
processes with jumps. These and similar examples of processes with jumps allow for jumps of
arbitrary size, and this does not mesh well with our model because we assume a finite state space
for X . Jump processes confined to a bounded domain have been defined (see, e.g., [2]) but
their structure is not very simple. For these technical reasons, we will present approximations to
processes similar to the stable process wrapped around a circle.
In both examples, we will identify R2 and C. Let S consist of N points on the unit circle D,
x j = exp( j2πi/N ), j = 1, . . . , N . We assume that the p j ’s are all equal to each other, hence,
p j = 1/N . In these examples, L takes values in R, not R2 .
Example 3.7. Consider a C 3 -function V : D → R. We write Vj = V (x j ). We define
¨
A( j, k) =
1
0
if x j and x k are adjacent on the unit circle,
otherwise.
For any pair of points x j and x k , not necessarily adjacent, we let
e jk (l) =
a
N2
2
(Vk − Vj )A( j, k)l +
1X
N
|(k − j) + nN |−1−α ,
n∈Z
where α ∈ (0, 2). We define
¨
a jk (l) =
e jk (l) ∨ 0
a
N 2 (Vk − Vj )A( j, k)l
ek j (l) > 0,
if a
otherwise.
Then
b jk (l) = N (Vk − Vj )A( j, k)l
626
Electronic Communications in Probability
and by (19) we have
vk = −N 2
X
Vk − Vj .
j:A(k, j)=1
Note that vk → ∆V (x) = V ′′ (x) when N → ∞ and x k → x.
Let PN be the distribution of (X , L) constructed above. Let W (x) = V (e i x ) and let (Z, S) be a
Markov process with the state space R × R and the following transition probabilities. The component Z is a jump process with the drift ∇W (Z)S = W ′ (Z)S. The jump density for the process Z is
Rt
P
−1−α
. We let S t = 0 ∆W (Zs )ds. Let Yt = exp(iZ t ) and P∗ be the distribution
n∈Z |(x − y) + n2π|
of (Y, S). We conjecture that PN → P∗ as N → ∞ and the process (Y, S) has the stationary distribution which is the product of the uniform measure on D and the standard normal distribution. The
process (Y, S) is a “stable process with index α, with inert drift, wrapped on the unit circle.”
R
Example 3.8. Consider a continuous function V : D → R with D V (x)d x = 0. Recall the notation
Vj = V (x j ). For any pair of points x j and x k , not necessarily adjacent, we let
e jk (l) =
a
1
1
N
2
(Vk − Vj )l +
X
!
|(k − j) + nN |
−1−α
,
n∈Z
where α ∈ (0, 2). We define
¨
a jk (l) =
e jk (l) ∨ 0
a
1
(Vk − Vj )l
N
ek j (l) > 0,
if a
otherwise.
Then b jk (l) = (1/N 2 )(Vk − Vj )l and by (19) we have
vk =
1
N
X
Vk − Vj .
1≤ j≤N , j6=k
Note that if arg x k → y when N → ∞ then vk → V (e i y ) −
R
D
V (x)d x = V (e i y ).
Let PN be the distribution of (X , L) constructed above. Let W (x) = V (e i x ) and let (Z, S) be a
Markov process with the state space R × R and the following transition probabilities.
The comRP
ponent Z is a jump process with the jump density f (x) = (W (x) − W ( y))s −
n∈Z ((x − y) +
Rt
n2π)−1−α at time t, given {Z t = y, S t = s}. We let S t = 0 W (Zs )ds. Let Yt = exp(iZ t ) and P∗ be
the distribution of (Y, S). We conjecture that PN → P∗ as N → ∞ and the process (Y, S) has the
stationary distribution which is the product of the uniform measure on D and the standard normal
distribution.
4 Acknowledgments
We are grateful to Zhenqing Chen and Tadeusz Kulczycki for very helpful suggestions.
Product-form stationary distribution
References
[1] R. Bass, K. Burdzy, Z.-Q. Chen and M. Hairer, Stationary distributions for diffusions with inert
drift Probab. Theory Rel. Fields (to appear)
[2] K. Bogdan, K. Burdzy and Z.-Q. Chen, Censored stable processes Probab. Theory Rel. Fields
127 (2003) 89–152. MR2006232
[3] K. Burdzy, R. Hołyst and Ł. Pruski, Brownian motion with inert drift, but without flux: a
model. Physica A 384 (2007) 278–284.
[4] K. Burdzy and D. White, A Gaussian oscillator. Electron. Comm. Probab. 9 (2004) paper 10,
pp. 92–95. MR2108855
[5] D. White, Processes with inert drift. Electron. J. Probab. 12 (2007) paper 55, pp. 1509–1546.
MR2365876
627
ELECTRONIC
COMMUNICATIONS
in PROBABILITY
MARKOV PROCESSES WITH PRODUCT-FORM STATIONARY DISTRIBUTION
KRZYSZTOF BURDZY1
Department of Mathematics, University of Washington, Seattle, WA 98195
email: burdzy@math.washington.edu
DAVID WHITE
Department of Mathematics, Belmont University, Nashville, TN 37212
email: white@u.washington.edu
Submitted November 3, 2007, accepted in final form November 18, 2008
AMS 2000 Subject classification: 60J25
Keywords: Markov process, stationary distribution, inert drift
Abstract
We consider a continuous time Markov process (X , L), where X jumps between a finite number of
states and L is a piecewise linear process with state space Rd . The process L represents an “inert
drift” or “reinforcement.” We find sufficient and necessary conditions for the process (X , L) to have
a stationary distribution of the product form, such that the marginal distribution of L is Gaussian.
We present a number of conjectures for processes with a similar structure but with continuous
state spaces.
1 Introduction
This research has been inspired by several papers on processes with inert drift [5, 4, 3, 1]. The
model involves a “particle” X and an “inert drift” L, neither of which is a Markov process by
itself, but the vector process (X , L) is Markov. It turns out that for some processes (X , L), the
stationary measure has the product form; see [1]. The first goal of this note is to give an explicit
characterization of all processes (X , L) with a finite state space for X and a product form stationary
distribution—see Theorem 2.1.
The second, more philosophical, goal of this paper is to develop a simple tool that could help
generate conjectures about stationary distributions for processes with continuous state space and
inert drift. So far, the only paper containing a rigorous result about the stationary distribution for
a process with continuous state space and inert drift, [1], was inspired by computer simulations.
Examples presented in Section 3 lead to a variety of conjectures that would be hard to arrive at
using pure intuition or computer simulations.
1
PARTIALLY SUPPORTED BY NSF GRANT DMS-0600206
614
Product-form stationary distribution
615
2 The model
Let S = {1, 2, . . . , N } for some integer N > 1 and let d ≥ 1 be an integer. We define a continuous
time Markov process (X (t), L(t)) on S × Rd as follows. We associate with each state j ∈ S a
vector v j ∈ Rd ,P
1 ≤ j ≤ N . Define L j (t) = µ({s ∈ [0, t] : X (s) = j}), where µ is Lebesgue measure,
and let L(t) = j∈S v j L j (t). To make the “reinforcement” non-trivial, we assume that at least one
of v j ’s is not 0. Since L will always belong to the hyperplane spanned by v j ’s, we also assume that
d = dim(span{v1 , . . . , vN }).
We also select non-negative functions ai j (l) which define the Poisson rates of jumps from state i
to j. The rates depend on l = L(t). We assume that ai j ’s are right-continuous with left limits.
Formally speaking, the process (X , L) is defined by its generator A as follows,
X
a ji (l)[ f (i, l) − f ( j, l)], j = 1, . . . , N , l ∈ Rd ,
Af ( j, l) = v j · ∇l f ( j, l) +
i6= j
for f : {1, . . . , N } × Rd → R.
We assume that (X , L) is irreducible in the sense of Harris, i.e., for some open set U ⊂ Rd and
some j0 ∈ S , for all (x, l) ∈ S × Rd , we have for some t > 0,
P((X (t), L(t)) ∈ { j0 } × U) > 0.
We are interested only in processes satisfying (14) below. Using that condition, it is easy to check
Harris irreducibility for each of our models by a direct argument. A standard coupling argument
shows that Harris irreducibility implies uniqueness of the stationary probability distribution (assuming existence of such).
The (formal) adjoint of A is given by
X
A∗ g( j, l) = −v j · ∇l g( j, l) +
[ai j (l)g(i, l) − a ji (l)g( j, l)], j = 1, . . . , N , l ∈ Rd .
(1)
i6= j
We are interested in invariant measures of product form so suppose that g( j, l) = p j g(l), where
R
P
j∈S p j = 1 and Rd g(l)d l = 1. We may assume that p j > 0 for all j; otherwise some points in
S are never visited. Under these assumptions, (1) becomes
X
[pi ai j (l)g(l) − p j a ji (l)g(l)], j = 1, . . . , N , l ∈ Rd .
A∗ g( j, l) = −p j v j · ∇g(l) +
i6= j
Theorem 2.1. Assume that for every i and j, the function l → ai j (l) is continuous. A probability
measure p j g(l)d jd l is invariant for the process (X , L) if and only if
X
− p j v j · ∇g(l) +
[pi ai j (l)g(l) − p j a ji (l)g(l)] = 0, j = 1, . . . , N , l ∈ Rd .
(2)
i6= j
Proof. Recall that the state space S for X is finite. Hence v∗ := sup j∈S |v j | < ∞. Fix arbitrary
r, t ∗ ∈ (0, ∞). It follows that,
sup
ai j (l) = a∗ < ∞.
i, j∈S ,l∈B(0,r+2t ∗ v∗ )
Note that we always have |L(t) − L(u)| ≤ v∗ |t − u|. Hence, if |L(0)| ≤ r + t ∗ v∗ and s, t > 0,
s + t ≤ t ∗ , then |L(s + t)| ≤ r + 2t ∗ v∗ and, therefore,
sup
j∈S ,u≤s+t
aX (u), j (L(u)) ≤ a∗ < ∞.
616
Electronic Communications in Probability
This implies that the probability of two or more jumps on the interval [s, s + t] is o(t). Assume
that |l| ≤ r + t ∗ v∗ and t ≤ t ∗ . Then we have the following three estimates. First,
P(X (t) = j | X (0) = i, L(0) = l) = ai j (l)t + R1i, j,l (t),
(3)
where the remainder R1i, j,l (t) satisfies supi, j∈S ,l∈B(0,t ∗ v∗ ) |R1i, j,l (t)| ≤ R1 (t) for some R1 (t) such that
lim t→0 R1 (t)/tP
= 0.
Let aii (l) = − j6=i ai j (l). We have
P(X (t) = i, L(t) = l + t vi | X (0) = i, L(0) = l) = 1 + aii (l)t + R2i,l (t),
(4)
where the remainder R2i,l (t) satisfies supi∈S ,l∈B(0,t ∗ v∗ ) |R2i,l (t)| ≤ R2 (t) for some R2 (t) such that
lim t→0 R2 (t)/t = 0.
Finally,
P(X (t) = i, L(t) 6= l + t vi | X (0) = i, L(0) = l) = R3i,l (t),
(5)
where the remainder R3i,l (t) satisfies supi∈S ,l∈B(0,t ∗ v∗ ) |R3i,l (t)| ≤ R3 (t) for some R3 (t) such that
lim t→0 R3 (t)/t = 0.
Now consider any C 1 function f ( j, l) with support in S × B(0, r). Recall that |L(t) − L(u)| ≤
v∗ |t − u|. Hence, Ei,l f (X t , L t ) = 0 for t ≤ t ∗ and |l| ≥ r + v∗ t ∗ .
Suppose that |l0 | ≤ r + v∗ t ∗ , t 1 ∈ (0, t ∗ ) and s ∈ (0, t ∗ − t 1 ). Then
Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l f (X t 1 , L t 1 )
Z
Z
X
X
=
f (k, r)P(X (t 1 + s) = k, L(t 1 + s) ∈ d r | X (t 1 ) = j, L(t 1 ) = l)
j∈S
Rd k∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
−
f ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d r | X (0) = i, L(0) = l0 ).
j∈S
Rd
We combine this formula with (3)-(5) to see that,
Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l f (X t 1 , L t 1 )
Z
X
X
=
( f (k, l) + O(s))(a jk (l)s + R1j,k,l (s))
j∈S
Rd k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
+
f ( j, l + sv j )(1 + a j j (l)s + R2j,l (s))
j∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
+
( f ( j, l) + O(s))R3j,l (s)
j∈S
Rd
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
−
f ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ),
j∈S
Rd
Product-form stationary distribution
617
which can be rewritten as
X
Z
( f ( j, l + sv j ) − f ( j, l))P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
j∈S
+
X
Z
f ( j, l + sv j )a j j (l) +
Rd
j∈S
X
f (k, l)a jk (l) s
k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
X
1
1
f (k, l)R j,k,l (s) + O(s)(a jk (l)s + R j,k,l (s))
+
Rd
j∈S
k∈S ,k6= j
+ f ( j, l
+ sv j )R2j,l (s) + ( f
( j, l) + O(s))R3j,l (s)
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ).
We will analyze the limit
1
lim (Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l0 f (X t 1 , L t 1 )).
s↓0 s
Note that
lim
s↓0
1X
s
Z
Rd
j∈S
+ f ( j, l
X
f
(k, l)R1j,k,l (s) + O(s)(a jk (l)s
k∈S ,k6= j
+ sv j )R2j,l (s) + ( f
+ R1j,k,l (s))
( j, l) + O(s))R3j,l (s)
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
= 0.
We also have
lim
s↓0
=
1X
s
X
Z
j∈S
( f ( j, l + sv j ) − f ( j, l))P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
Z
∇l f ( j, l) · v j P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ),
Rd
j∈S
and
lim
s↓0
1X
s
j∈S
Z
f ( j, l + sv j )a j j (l) +
Rd
X
f (k, l)a jk (l) s
k∈S ,k6= j
× P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Z
X
X
=
f (k, l)a jk (l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ).
j∈S
Rd k∈S
618
Electronic Communications in Probability
This implies that
d
dt
=
¯
¯
Ei,l0 f (X t , L t )¯
X
j∈S
=
X
j∈S
1
= lim (Ei,l0 f (X t 1 +s , L t 1 +s ) − Ei,l0 f (X t 1 , L t 1 ))
s↓0 s
t=t 1
!
X
∇l f ( j, l) · v j +
f (k, l)a jk (l) P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 ) (6)
Z
Rd
k∈S
Z
Af ( j, l)P(X (t 1 ) = j, L(t 1 ) ∈ d l | X (0) = i, L(0) = l0 )
Rd
= Ei,l0 Af (X t 1 , L t 1 ),
(7)
for all i and |l0 | ≤ r + v∗ t ∗ .
We will argue that
P((X (t 1 ), L(t 1 )) ∈ · | X (0) = i, L(0) = l) → P((X (t 1 ), L(t 1 )) ∈ · | X (0) = i, L(0) = l0 ),
weakly when l → l0 . Let Tk be the time of the k-th jump of X . We have
Z t
P(T1 > t | X (0) = i, L(0) = l) = exp
(8)
aii (l + svi )ds .
0
Since l → aii (l) is continuous, we conclude that
P(T1 > t | X (0) = i, L(0) = l) → P(T1 > t | X (0) = i, L(0) = l0 ),
weakly as l → l0 . This and continuity of l → ai j (l) for every j implies that
P((X (T1 ), L(T1 )) ∈ · | X (0) = i, L(0) = l) → P((X (T1 ), L(T1 )) ∈ · | X (0) = i, L(0) = l0 ),
weakly when l → l0 . By the strong Markov property applied at Tk ’s, we obtain inductively that
P((X (Tk ), L(Tk )) ∈ · | X (0) = i, L(0) = l) → P((X (Tk ), L(Tk )) ∈ · | X (0) = i, L(0) = l0 ),
when l → l0 , for every k ≥ 1. This easily implies (8), because the number of jumps is stochastically
bounded on any finite interval.
P
Since ( j, l) ,→ ∇l f ( j, l) · v j +¯ k∈S f (k, l)a jk (l) is a continuous function, it follows from (6) and
¯
is continuous on the set |l0 | ≤ r + v∗ t ∗ .
(8) that l0 ,→ ddt Ei,l0 f (X t , L t )¯
t=t 1
¯
¯
Recall that Ei,l0 f (X t , L t ) = 0 for t ≤ t ∗ and |l0 | ≥ r + v∗ t ∗ . Hence, l0 → ddt Ei,l0 f (X t , L t )¯
is
t=t 1
continuous for all t 1 ≤ t ∗ and all values of i.
Fix some t ≤ t ∗ and let u t ( j, l) = E j,l f (X t , L t ). We have just shown that for a fixed t ≤ t ∗ and any
j, the function l → u t ( j, l) is C 1 . Hence we can apply (7) with f ( j, l) = u t ( j, l) to obtain,
1
E j,l f (X t , L t ) = lim (E j,l f (X t+s , L t+s ) − E j,l f (X t , L t ))
s↓0 s
dt
1
= lim (E j,l u t (X s , Ls ) − u t ( j, l))
s↓0 s
= (Au t )( j, l).
d
(9)
Product-form stationary distribution
619
P
Since sup j,l ∇l f ( j, l) · v j + k∈S f (k, l)a jk (l) < ∞, formula (6) shows that
d
sup
E j,l f (X s , Ls ) < ∞.
dt
j,l,s≤t ∗
(10)
Now assume that (2) is true and let π(d j, d l) = p j g(l)d jd l. In view of (10), we can change the
order of integration in the following calculation. For 0 ≤ t 1 < t 2 ≤ t ∗ , using (9),
Eπ f (X (t 2 ), L(t 2 )) − Eπ f (X (t 1 ), L(t 1 ))
Z
Z
X
X
E j,l f (X t 2 , L t 2 )p j g(l)d l −
=
j∈S
=
X
j∈S
Z
t2
=
Z
t1
t2
=
t1
Rd
Z
j∈S
Z
Rd
X
j∈S
X
j∈S
t2
t1
Z
d
ds
d
Rd
ds
(11)
E j,l f (X t 1 , L t 1 )p j g(l)d l
Rd
E j,l f (X s , Ls )dsp j g(l)d l
E j,l f (X s , Ls )p j g(l)d l ds
Z
(Aus )( j, l)p j g(l)d l ds.
Rd
Let h( j, l) = p j g(l). For a fixed j and s ≤ t ∗ , the function us ( j, l) = 0 outside a compact set, so we
can use integration by parts to show that
Z
Z
X
X
us ( j, l)(A∗ h)( j, l)d l.
(12)
(Aus )( j, l)p j g(l)d l =
j∈S
Rd
j∈S
Rd
We combine this with the previous formula and the assumption that A∗ h ≡ 0 to see that
Z t2
Z
X
Eπ f (X (t 2 ), L(t 2 )) − Eπ f (X (t 1 ), L(t 1 )) =
us ( j, l)(A∗ h)( j, l)d l ds = 0.
t1
j∈S
Rd
It follows that t → Eπ f (X (t), L(t)) is constant for every C 1 function f ( j, l) with compact support.
This proves that the distributions of (X (t 1 ), L(t 1 )) and (X (t 2 ), L(t 2 )) are identical under π, for all
0 ≤ t1 < t2 ≤ t∗.
Conversely, assume that π(d j, d l) = p j g(l)d jd l is invariant. Then the left hand side of (11) is
zero for all 0 ≤ t 1 < t 2 ≤ t ∗ . This implies that
Z
X
(Aus )( j, l)p j g(l)d l = 0
j∈S
Rd
for a set of s that is dense on [0, ∞). By (12),
Z
X
us ( j, l)(A∗ h)( j, l)d l = 0
j∈S
(13)
Rd
for a set of s that is dense on [0, ∞). Note that lims↓0 us ( j, l) = f ( j, l). Hence, the collection of
C 1 functions us ( j, l), obtained by taking arbitrary C 1 functions f ( j, l) with compact support and
positive reals s dense in [0, ∞), is dense in the family of C 1 functions with compact support. This
and (13) imply that A∗ h ≡ 0, that is, (2) holds.
620
Electronic Communications in Probability
Corollary 2.2. If a probability measure p j g(l)d jd l is invariant for the process (X , L) then
X
p j v j = 0.
(14)
j∈S
Proof. Summing (2) over j, we obtain
X
−p j v j · ∇g(l) = 0,
(15)
j∈S
for all l. Since g is integrable over Rd , it is standard to show that there exist l1 , l2 , . . . , l d which
span Rd . Applying (15) to all l1 , l2 , . . . , we obtain (14).
It will be convenient to use the following notation,
bi j (l) = pi ai j (l) − p j a ji (l).
(16)
Note that bi j = −b ji .
Corollary 2.3. A probability measure p j g(l)d jd l is invariant for the process (X , L) and g(l) is the
Gaussian density
g(l) = (2π)−d/2 exp(−|l|2 /2),
if and only if the following equivalent conditions hold,
X
[pi ai j (l) − p j a ji (l)] = 0,
p j vj · l +
i6= j
p j vj · l +
X
bi j (l) = 0,
(17)
j = 1, . . . , N , l ∈ Rd ,
j = 1, . . . , N , l ∈ Rd .
(18)
(19)
i6= j
Proof. If g(l) is the Gaussian density then ∇g(l) = −l g(l) and (2) is equivalent to (18). Conversely, if (2) and (18) are satisfied then ∇g(l) = −l g(l), so g(l) must have the form (17).
In the rest of the paper we will consider only processes satisfying (18)-(19).
Example 2.4. We now present some choices for ai j ’s. Recall the notation x + = max(x, 0), x − =
− min(x, 0), and the fact that x + − x − = x. Given v j ’s, p j ’s and bi j ’s which satisfy (19) and the
condition bi j = −b ji , we may take
+
ai j (l) = bi j (l) /pi .
(20)
Then
so
+
+
−
a ji (l) = b ji (l) /p j = −bi j (l) /p j = bi j (l) /p j ,
+
−
pi ai j (l) − p j a ji (l) = bi j (l) − bi j (l) = bi j (l),
as desired.
The above is a special case, in a sense, of the following. Suppose that p j = pi for all i and j.
Assume that v j ’s and bi j ’s satisfy (19) and the condition bi j = −b ji . Fix some c > 0 and let
ai j (l) =
bi j (l) exp(c bi j (l))
exp(c bi j (l)) − exp(−c bi j (l))
.
(21)
It is elementary to check that with this definition, (16) is satisfied for all i and j, because bi j (l) =
−b ji (l). The formula (21) arose naturally in [5]. Note that (20) (with all pi ’s equal) is the limit
of (21) as c → ∞.
Product-form stationary distribution
621
3 Approximation of processes with continuous state
space
This section is contains examples of processes (X , L) with finite state space for X , and conjectures
concerned with processes with continuous state space. There are no proofs in this section
First we will consider processes that resemble diffusions with reflection. In these models, the
“inert drift” is accumulated only at the “boundary” of the domain.
We will now assume that elements of S are points in a Euclidean space Rn with n ≤ N . We denote
them S = {x 1 , x 2 , . . . , x N }. In other words, by abuse of notation, we switch from j to x j . We also
take v j ∈ Rn , i.e. d = n. Moreover, we limit ourselves to functions bi j (l) of the form bi j · l for some
vector bi j ∈ Rn . Then (19) becomes
0 + b12 + b13 + · · · + b1N = −p1 v1
−b12 − 0
+ b23 + · · · + b2n = −p2 v2
(22)
...
−b1N − b2N − b3N − · · · − 0
= −p N vN .
Consider any orthogonal transformation Λ : Rn → Rn . If {bi j , v j , p j } satisfy (22) then so do
{Λbi j , Λv j , p j }.
Suppose that ai j (l) have the form ai j · l for some ai j ∈ Rn . If {ai j , v j , p j } satisfy (18) then so do
{Λai j , Λv j , p j }. Moreover, the process with parameters {ai j , v j } has the same transition probabilities as the one with parameters {Λai j , Λv j }.
Example 3.1. Our first example is a reflected random walk on the interval [0, 1]. Let x j =
( j − 1)/(N − 1) for j = 1, . . . , N . We will construct a process with all p j ’s equal to each other, i.e.,
p j = 1/N . We will take l ∈ R1 , v1 = α and vN = −α, for some α = α(N ) > 0, and all other v j = 0,
so that the “inert drift” L changes only at the endpoints of the interval. We also allow jumps only
between adjacent points, so bi j = 0 for |i − j| > 1. Then (22) yields
b12 = −α/N
−b12 + b23 = 0
···
−b(N −1)N = α/N .
Solving this, we obtain bi(i+1) = −α/N for all i.
We would like to find a family of semi-discrete models indexed by N that would converge to a
continuous process with product-form stationary distribution as N → ∞. For 1 < i < N , we set
ai(i+1) (l) = AR (l, N ) and a(i+1)i (l) = A L (l, N ). We would like the random walk to have variance of
order 1 at time 1, for large N , so we need
AR + A L = N 2 .
(23)
Since bi(i+1) = −α/N for all i, AR and A L have to satisfy
A L − AR = αl.
(24)
When l is of order 1, we would like to have drift of order 1 at time 1, so we take α = N . Then
(24) becomes
A L − AR = N l.
(25)
622
Electronic Communications in Probability
Solving (23) and (25) gives
AL =
N2 + Nl
2
AR =
,
N2 − Nl
2
.
Unfortunately, A L and AR given by the above formula can take negative values—this is not allowed
because ai j ’s have to be positive. However, for every N , the stationary distribution of L is standard
normal, so l typically takes values of order 1. We are interested in large N so, intuitively speaking,
AR and A L are not likely to take negative values. To make this heuristics rigorous, we modify the
formulas for AR and A L as follows,
AL =
N2 + Nl
2
∨ 0 ∨ N l,
AR =
N2 − Nl
2
∨ 0 ∨ (−N l).
(26)
Let PN denote the distribution of (X , L) with the above parameters. We conjecture that as N → ∞,
PN converge to the distribution of reflected Brownian motion in [0, 1] with inert drift, as defined
in [5, 1]. The stationary distribution for this continuous time process is the product of the uniform
measure in [0, 1] and the standard normal; see [1].
Example 3.2. This example is a semi-discrete approximation to reflected Brownian motion in a
bounded Euclidean subdomain of Rn , with inert drift. In this example we proceed in the reversed
order, starting with bi j ’s and ai j ’s.
Consider an open bounded connected set D ⊂ Rn . Let K be a (large) integer and let DK = Zn /K∩D,
i.e., DK is the subset of the square lattice with mesh 1/K that is inside D. We assume that DK is
connected, i.e., any vertices in DK are connected by a path in DK consisting of edges of length
1/K. We take S = DK and l ∈ Rn .
We will consider nearest neighbor random walk, i.e., we will take ai j (l) = 0 for |x i − x j | > 1/K.
In analogy to (26), we define
ai j (l) =
K2
2
(1 + (x i − x j ) · l) ∨ 0 ∨ K 2 (x i − x j ) · l.
(27)
Then bi j (l) = (K 2 /N )(x i − x j ) · l. Let us call a point in S = DK an interior point if it has 2n
neighbors in DK . We now define v j ’s using (22) with p j = 1/|DK |. For all interior points x j , the
vector v j is 0, by symmetry. For all boundary (that is, non-interior) points x j , the vector v j is not
0.
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. We conjecture that as K → ∞, PK converge to the distribution of normally reflected
Brownian motion in D with inert drift, as defined in [5, 1]. If D is C 2 then it is known that the
stationary distribution for this continuous time process is the product of the uniform measure in
D and the standard Gaussian distribution; see [1].
The next two examples are discrete counterparts of processes with continuous state space and
smooth inert drift. The setting is similar to that in Example 3.2. We consider an open bounded
connected set D ⊂ Rn . Let K be a (large) integer and let DK = Zn /K ∩ D, i.e., DK is the subset
of the square lattice with mesh 1/K that is inside D. We assume that DK is connected, i.e., any
vertices in DK are connected by a path in DK consisting of edges of length 1/K. We take S = DK
and l ∈ Rn .
Product-form stationary distribution
623
Example 3.3. This example is concerned with a situation when the stationary distribution has the
form p j g(l) where p j ’s are not necessarily equal. We start with a C 2 “potential” V : D → R. We
will write Vj instead of V (x j ). Let p j = c exp(−Vj ). We need an auxiliary function
di j =
2(pi − p j )
pi (Vj − Vi ) − p j (Vi − Vj )
.
Note that di j = d ji and for a fixed i, we have di jK → 1 when K → ∞ and |i − jK | = 1/K.
Let ai j (l) = 0 for |x i − x j | > 1/K, and for |x i − x j | = 1/K,
ei j (l) =
a
We set
¨
ai j (l) =
K2
2
(2 + di j (Vi − Vj ) + (x j − x i ) · l).
ei j (l) ∨ 0
a
(K 2 /2pi )(pi + p j )(x j − x i ) · l
e ji (l) > 0,
if a
otherwise.
(28)
e ji (l) > 0 and a
ei j (l) > 0 then
If a
bi j (l) = pi ai j (l) − p j a ji (l)
=
=
=
K2
2
K2
2
K2
2
(2(pi − p j ) + (pi (Vi − Vj ) − p j (Vj − Vi ))di j + (pi (x j − x i ) − p j (x i − x j )) · l)
(2(pi − p j ) − 2(pi − p j ) + (pi + p j )(x j − x i ) · l)
(pi + p j )(x j − x i ) · l.
e ji (l) ≤ 0 or a
ei j (l) ≤ 0. Consider an
It follows from (28) that the above formula holds also if a
interior point x j . For (19) to be satisfied, we have to take
vj = −
X
1
pj
|x i −x j |=1/K
K2
2
(pi + p j )(x j − x i ).
For large K, series expansion shows that
v j ≈ −∇V.
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. We recall the following SDE from [1],
d Yt = −∇V (Yt ) d t + S t d t + d B t ,
dS t = −∇V (Yt ) d t ,
where B is standard n-dimensional Brownian motion and V is as above. Let P∗ denote the distribution of (Y, S). We conjecture that as K → ∞, PK converge to P∗ . Under mild assumptions on V ,
it is known that the stationary distribution for (Y, S) is the product of the measure exp(−V (x))d x
and the standard Gaussian distribution; see [1].
624
Electronic Communications in Probability
Example 3.4. We again consider the situation when all p j ’s are equal, i.e., p j = 1/N . Consider a
C 2 function V : D → R. We let ai j (l) = 0 for |x i − x j | > 1/K. If |x i − x j | = 1/K, we let
ei j (l) =
a
We set
¨
ai j (l) =
K2
2
(1 + (Vj + Vi )(x j − x i ) · l).
ei j (l) ∨ 0
a
K 2 (Vj + Vi )(x j − x i ) · l
e ji (l) > 0,
if a
otherwise.
(29)
Then bi j (l) = (1/N )K 2 (Vj + Vi )(x j − x i ) · l and
X
vj = K 2
(Vj + Vi )(x j − x i ).
|x i −x j |=1/K
For large K, we have v j ≈ −2∇V .
Fix D ⊂ Rn and consider large K. Let PK denote the distribution of (X , L) constructed in this
example. Consider the following SDE,
d Yt = V (Yt )S t d t + d B t ,
dS t = −2∇V (Yt ) d t ,
where B is standard n-dimensional Brownian motion and V is as above. Let P∗ denote the distribution of (Y, S). We conjecture that as K → ∞, PK converge to P∗ , and that the stationary distribution
for (Y, S) is the product of the uniform measure on D and the standard Gaussian distribution.
The next example and conjecture are devoted to examples where the inert drift is related to the
curvature of the state space, in a suitable sense.
Example 3.5. In this example, we will identify R2 and C. The imaginary unit will be denoted by i,
as usual. Let S consist of N points on a circle with radius r > 0, x j = r exp( j2πi/N ), j = 1, . . . , N .
We assume that the p j ’s are all equal to each other.
For any pair of adjacent points x j and x k , we let
e jk (l) =
a
and
¨
a jk (l) =
N2
2
(1 + (x k − x j ) · l),
e jk (l) ∨ 0
a
N 2 (x k − x j ) · l
ek j (l) > 0,
if a
otherwise,
with the other ak j (l) = 0. Then b j( j+1) = N (x j+1 − x j ) · l, and by (19) we have
v j = N 2 (x j−1 − x j ) + N 2 (x j+1 − x j ) = 2N 2 (cos(2π/N ) − 1)x j .
Note that v j → −4π2 x j when N → ∞.
Let PN be the distribution of (X , L) constructed above.
Let C be the circle with radius r > 0 and center 0, and let T y be the projection of R2 onto the
tangent line to C at y ∈ C . Consider the following SDE,
d Yt = TYt (S t ) d t + d B t ,
dS t = −4π2 Yt d t ,
Product-form stationary distribution
625
where Y takes values in C and B is Brownian motion on this circle. Let P∗ be the distribution of
(Y, S). We conjecture that as N → ∞, PN converge to P∗ , and that the stationary distribution for
(Y, S) is the product of the uniform measure on the circle and the standard Gaussian distribution.
Conjecture 3.6. We propose a generalization of the conjecture stated in the previous example.
We could start with an explicit discrete approximation, just like in other examples discussed so far.
The notation would be complicated and the whole procedure would not be illuminating, so we
skip the approximation and discuss only the continuous model.
Let S ⊂ Rn be a smooth (n − 1)-dimensional surface, let T y be the projection of Rn onto the
tangent space to S at y ∈ S , let n( y) be the inward normal to S at y ∈ S , and let ρ be the
mean curvature at y ∈ S . Consider the following SDE,
d Yt = TYt (S t ) d t + d B t ,
dS t = c0 ρ −1 n(Yt ) d t ,
where Y takes values in S and B is Brownian motion on this surface. We conjecture that for some
c0 depending only on the dimension n, the stationary distribution for (Y, S) exists, is unique and
is the product of the uniform measure on S and the standard Gaussian distribution.
We end with examples of processes that are discrete approximations of continuous-space processes
with jumps. It is not hard to construct examples of discrete-space processes that converge in
distribution to continuous-space processes with jumps. Stable processes are a popular family of
processes with jumps. These and similar examples of processes with jumps allow for jumps of
arbitrary size, and this does not mesh well with our model because we assume a finite state space
for X . Jump processes confined to a bounded domain have been defined (see, e.g., [2]) but
their structure is not very simple. For these technical reasons, we will present approximations to
processes similar to the stable process wrapped around a circle.
In both examples, we will identify R2 and C. Let S consist of N points on the unit circle D,
x j = exp( j2πi/N ), j = 1, . . . , N . We assume that the p j ’s are all equal to each other, hence,
p j = 1/N . In these examples, L takes values in R, not R2 .
Example 3.7. Consider a C 3 -function V : D → R. We write Vj = V (x j ). We define
¨
A( j, k) =
1
0
if x j and x k are adjacent on the unit circle,
otherwise.
For any pair of points x j and x k , not necessarily adjacent, we let
e jk (l) =
a
N2
2
(Vk − Vj )A( j, k)l +
1X
N
|(k − j) + nN |−1−α ,
n∈Z
where α ∈ (0, 2). We define
¨
a jk (l) =
e jk (l) ∨ 0
a
N 2 (Vk − Vj )A( j, k)l
ek j (l) > 0,
if a
otherwise.
Then
b jk (l) = N (Vk − Vj )A( j, k)l
626
Electronic Communications in Probability
and by (19) we have
vk = −N 2
X
Vk − Vj .
j:A(k, j)=1
Note that vk → ∆V (x) = V ′′ (x) when N → ∞ and x k → x.
Let PN be the distribution of (X , L) constructed above. Let W (x) = V (e i x ) and let (Z, S) be a
Markov process with the state space R × R and the following transition probabilities. The component Z is a jump process with the drift ∇W (Z)S = W ′ (Z)S. The jump density for the process Z is
Rt
P
−1−α
. We let S t = 0 ∆W (Zs )ds. Let Yt = exp(iZ t ) and P∗ be the distribution
n∈Z |(x − y) + n2π|
of (Y, S). We conjecture that PN → P∗ as N → ∞ and the process (Y, S) has the stationary distribution which is the product of the uniform measure on D and the standard normal distribution. The
process (Y, S) is a “stable process with index α, with inert drift, wrapped on the unit circle.”
R
Example 3.8. Consider a continuous function V : D → R with D V (x)d x = 0. Recall the notation
Vj = V (x j ). For any pair of points x j and x k , not necessarily adjacent, we let
e jk (l) =
a
1
1
N
2
(Vk − Vj )l +
X
!
|(k − j) + nN |
−1−α
,
n∈Z
where α ∈ (0, 2). We define
¨
a jk (l) =
e jk (l) ∨ 0
a
1
(Vk − Vj )l
N
ek j (l) > 0,
if a
otherwise.
Then b jk (l) = (1/N 2 )(Vk − Vj )l and by (19) we have
vk =
1
N
X
Vk − Vj .
1≤ j≤N , j6=k
Note that if arg x k → y when N → ∞ then vk → V (e i y ) −
R
D
V (x)d x = V (e i y ).
Let PN be the distribution of (X , L) constructed above. Let W (x) = V (e i x ) and let (Z, S) be a
Markov process with the state space R × R and the following transition probabilities.
The comRP
ponent Z is a jump process with the jump density f (x) = (W (x) − W ( y))s −
n∈Z ((x − y) +
Rt
n2π)−1−α at time t, given {Z t = y, S t = s}. We let S t = 0 W (Zs )ds. Let Yt = exp(iZ t ) and P∗ be
the distribution of (Y, S). We conjecture that PN → P∗ as N → ∞ and the process (Y, S) has the
stationary distribution which is the product of the uniform measure on D and the standard normal
distribution.
4 Acknowledgments
We are grateful to Zhenqing Chen and Tadeusz Kulczycki for very helpful suggestions.
Product-form stationary distribution
References
[1] R. Bass, K. Burdzy, Z.-Q. Chen and M. Hairer, Stationary distributions for diffusions with inert
drift Probab. Theory Rel. Fields (to appear)
[2] K. Bogdan, K. Burdzy and Z.-Q. Chen, Censored stable processes Probab. Theory Rel. Fields
127 (2003) 89–152. MR2006232
[3] K. Burdzy, R. Hołyst and Ł. Pruski, Brownian motion with inert drift, but without flux: a
model. Physica A 384 (2007) 278–284.
[4] K. Burdzy and D. White, A Gaussian oscillator. Electron. Comm. Probab. 9 (2004) paper 10,
pp. 92–95. MR2108855
[5] D. White, Processes with inert drift. Electron. J. Probab. 12 (2007) paper 55, pp. 1509–1546.
MR2365876
627