M01439
A Comparison of MCMC Samplers for
Estimating Leveraged Stochastic Volatility
Model
Didit B. Nugroho1,2 and Takayuki Morimoto1
1
2
Department of Mathematical Sciences, Kwansei Gakuin University, Japan
Department of Mathematics, Satya Wacana Christian University, Indonesia
December 23, 2013, Hiroshima University of Economics
Abstract
This study describes two Hamiltonian Monte Carlo (HMC)-based methods for estimating leveraged
stochastic volatility model and compares efficiency of these methods to the multi-move MetropolisHastings (MM-MH) which has been previouly proposed. Employing daily TOPIX returns, the Riemann manifold HMC sampler provides the best efficiency in terms of autocorrelation time, followed
by the MM-MH and HMC samplers.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
1. Introduction: SV and MCMC methods
Stochastic volatility (SV) model is among the most important tools
for modeling volatility of a financial time series.
A problem in parametric SV models is that it is not possible to obtain
an explicit expression for the likelihood function of some unknown
parameters.
An approach has become very attractive is Markov Chain Monte Carlo
(MCMC) method proposed by Shephard (1993) and Jacquier et al
(1994).
Omori and Watanabe (2008) proposed an efficient multi-move (block)
Metropolis-Hastings sampler for sampling high-dimensional latent volatility in the leveraged SV model.
Recently, an alternative efficient sampling was obtained using Hamiltonian Monte Carlo (HMC) and Riemann manifold HMC (RMHMC)
introduced by Duane et al. (1987) and Girolami and Calderhead
(2011), respectively.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
2. Survey on MCMC Comparison
Kim et al. (1998) and Jacquier-Polson (2011) compared the performance of the single-move and multi-move MH (MM-MH) samplers to
estimate basic SV model. The MM-MH sampler has been proposed
to reduce sample autocorrelations effectively. Jacquier-Polson particularly note that the single-move and multi-move samplers deliver
almost the same output.
Takaishi (2009) compared the performance of the single-move MH
and HMC samplers and concluded that HMC sampler is superior to
the single-move sampler.
Girolami and Calderhead (2011) showed that the RMHMC sampler
yields the best performance for estimating the basic SV model among
four HMC-based samplers, in terms of normalized effective sample
size.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
3. Our Purpose
First purpose of this study is to extend the HMC and RMHMC sampling procedures proposed by Girolami and Calderhead (2011) for the
leveraged SV model.
Second purpose is to compare the performance of the MM-MH, HMC,
and RMHMC samplers for estimating the leveraged SV model, in
terms of integrated autocorrelation time (IACT).
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
4. HMC and RMHMC Samplers
HMC-based methods are based on Hamiltonian dynamics system:
n
o
H (θ, ω ) = −L(θ) + 12 log (2π )D |M| + 21 ω ′ M−1 ω,
where L(θ) is the logarithm of the joint probability distribution for
the parameters θ ∈ RD , M is the covariance matrix, and ω ∈ RD is
the independent auxiliary variable.
In the RMHMC sampling, M depends on the variable θ and is chosen
to be the metric tensor, i.e.
2
∂
M(θ) = −Ey|θ
L(
θ
)
∂θ2
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
4. HMC and RMHMC Samplers (Cont’ed)
The full algorithm for HMC or RMHMC can then be summarized in the
following three steps.
(1) Randomly draw a sample momentum vector ω ∼ N (ω |0, M).
(2) Run the leapfrog algorithm for NL steps with step size ∆τ to
generate a proposal (θ∗ , ω ∗ ) according to the Hamiltonian equations
∂H
dθ
=
dτ
∂ω
and
dω
∂H
=−
.
dτ
∂θ
over a fictitious time τ.
(3) Accept (θ∗ , ω ∗ ) with probability
P(θ, ω; θ∗ , ω ∗ ) = min {1, exp{−H (θ∗ , ω ∗ ) + H (θ, ω )}} .
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
5. Leveraged SV Model
Omori and Watanabe (2008) proposed a leveraged SV model with
normal errors (LSV-N model, hereafter) formulated as
1
Rt = σr e 2 ht ǫt ,
t = 1, . . . , T
ht+1 = φht+ σh vt ,
t = 1, . . . , T − 1
σh2
.
h1 ∼ N 0, (1−φ2
0
1 ρ
ǫt
∼ N
,
vt
0
ρ 1
They estimate both the parameter φ and parameter vector (σr , σh , ρ)
using MH algorithm separately and the latent volatility ht using multimove MH algorithm.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
6. Comparison on Real Data
The LSV model is fitted to the daily Japanese stock data used by Omori
and Watanabe (2008), that is TOPIX from August 1, 1997 to July 31,
2002.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
6. Comparison on Real Data (Cont’ed)
Tabel: Tuning parameters for the HMC and RMHMC implementations.
Sampler
HMC
RMHMC
Parameter of
sampler
h
NL
∆τ
NL
∆τ
NFPI
100
0.01
50
0.1
-
Parameter of model
φ
(σr , σh , ρ)
100
0.0125
6
0.5
5
Note: NFPI denotes the number of fixed point iterations.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
100
0.0125
6
0.5
5
6. Comparison on Real Data (Cont’ed)
Tabel: Posterior summary statistics.
Parameter
Mean (SD)
95% Interval
Panel A: Using MM-MH sampler
σr
1.259 (0.070) [1.121, 1.398]
φ
0.945 (0.019) [0.902, 0.974]
σh
0.193 (0.033) [0.138, 0.267]
ρ
−0.442 (0.103) [−0.630, −0.231]
Panel B: Using HMC sampler
σr
1.241 (0.070) [1.098, 1.377]
φ
0.954 (0.014) [0.925, 0.980]
σh
0.169 (0.025) [0.120, 0.219]
ρ
−0.461 (0.097) [−0.648, −0.272]
Panel C: Using RMHMC sampler
σr
1.238 (0.070) [1.098, 1.378]
φ
0.953 (0.014) [0.923, 0.980]
σh
0.171 (0.026) [0.122, 0.222]
ρ
−0.456 (0.098) [−0.636, −0.259]
D.B. Nugroho & T. Morimoto
IACT
Time (sec)
20.8
118.2
206.7
92.7
−
53.0
132.7
279.9
83.9
465.88
19.4
109.4
185.0
69.8
325.41
Comparison of MCMC Samplers
6. Comparison on Real Data (Cont’ed)
Tabel: Acceptance rates.
Parameter
h
φ
(σr , σh , ρ)
Acceptance rates
HMC
RMHMC
MM-MH
0.856
0.889
0.975
0.955
0.910
0.901
0.990
0.952
0.966
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
7. Conclusion
Based on the empirical results, we conclude:
The RMHMC sampler give the best performance in terms of autocorrelation time.
In particular, the HMC sampler exhibits slightly slower convergence
than MM-HMC sampler except for the leverage parameter.
Regarding the estimates of parameters, all samplers give similar estimates.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
Some References
Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and
Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society, Series
B, 73 (2), 1–37.
Jacquier, E. and Polson, N. G. (2011). Bayesian methods in finance. In J.
Geweke, G. Koop, and H. van Dijk (Eds.), Handbook of Bayesian Econometrics.
Oxford University Press.
Kim, S. and Shephard, N. and Chib, S. (1998). Stochastic volatility: likelihood
inference and comparison with ARCH models. In N. Shephard (Ed.), Stochastic
Volatility: Selected Readings. Oxford University Press.
Omori, Y. and Watanabe, T. (2008). Block sampler and posterior mode estimation for asymmetric stochastic volatility models. Computational Statistics and
Data Analysis, 52, 2892–2910.
Takaishi, T. (2009). Bayesian Inference of stochastic volatility by Hybrid Monte
Carlo. Journal of Circuits, Systems, and Computers, 18 (8), 1381–1396.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
Thanks for your attention!
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
Estimating Leveraged Stochastic Volatility
Model
Didit B. Nugroho1,2 and Takayuki Morimoto1
1
2
Department of Mathematical Sciences, Kwansei Gakuin University, Japan
Department of Mathematics, Satya Wacana Christian University, Indonesia
December 23, 2013, Hiroshima University of Economics
Abstract
This study describes two Hamiltonian Monte Carlo (HMC)-based methods for estimating leveraged
stochastic volatility model and compares efficiency of these methods to the multi-move MetropolisHastings (MM-MH) which has been previouly proposed. Employing daily TOPIX returns, the Riemann manifold HMC sampler provides the best efficiency in terms of autocorrelation time, followed
by the MM-MH and HMC samplers.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
1. Introduction: SV and MCMC methods
Stochastic volatility (SV) model is among the most important tools
for modeling volatility of a financial time series.
A problem in parametric SV models is that it is not possible to obtain
an explicit expression for the likelihood function of some unknown
parameters.
An approach has become very attractive is Markov Chain Monte Carlo
(MCMC) method proposed by Shephard (1993) and Jacquier et al
(1994).
Omori and Watanabe (2008) proposed an efficient multi-move (block)
Metropolis-Hastings sampler for sampling high-dimensional latent volatility in the leveraged SV model.
Recently, an alternative efficient sampling was obtained using Hamiltonian Monte Carlo (HMC) and Riemann manifold HMC (RMHMC)
introduced by Duane et al. (1987) and Girolami and Calderhead
(2011), respectively.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
2. Survey on MCMC Comparison
Kim et al. (1998) and Jacquier-Polson (2011) compared the performance of the single-move and multi-move MH (MM-MH) samplers to
estimate basic SV model. The MM-MH sampler has been proposed
to reduce sample autocorrelations effectively. Jacquier-Polson particularly note that the single-move and multi-move samplers deliver
almost the same output.
Takaishi (2009) compared the performance of the single-move MH
and HMC samplers and concluded that HMC sampler is superior to
the single-move sampler.
Girolami and Calderhead (2011) showed that the RMHMC sampler
yields the best performance for estimating the basic SV model among
four HMC-based samplers, in terms of normalized effective sample
size.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
3. Our Purpose
First purpose of this study is to extend the HMC and RMHMC sampling procedures proposed by Girolami and Calderhead (2011) for the
leveraged SV model.
Second purpose is to compare the performance of the MM-MH, HMC,
and RMHMC samplers for estimating the leveraged SV model, in
terms of integrated autocorrelation time (IACT).
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
4. HMC and RMHMC Samplers
HMC-based methods are based on Hamiltonian dynamics system:
n
o
H (θ, ω ) = −L(θ) + 12 log (2π )D |M| + 21 ω ′ M−1 ω,
where L(θ) is the logarithm of the joint probability distribution for
the parameters θ ∈ RD , M is the covariance matrix, and ω ∈ RD is
the independent auxiliary variable.
In the RMHMC sampling, M depends on the variable θ and is chosen
to be the metric tensor, i.e.
2
∂
M(θ) = −Ey|θ
L(
θ
)
∂θ2
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
4. HMC and RMHMC Samplers (Cont’ed)
The full algorithm for HMC or RMHMC can then be summarized in the
following three steps.
(1) Randomly draw a sample momentum vector ω ∼ N (ω |0, M).
(2) Run the leapfrog algorithm for NL steps with step size ∆τ to
generate a proposal (θ∗ , ω ∗ ) according to the Hamiltonian equations
∂H
dθ
=
dτ
∂ω
and
dω
∂H
=−
.
dτ
∂θ
over a fictitious time τ.
(3) Accept (θ∗ , ω ∗ ) with probability
P(θ, ω; θ∗ , ω ∗ ) = min {1, exp{−H (θ∗ , ω ∗ ) + H (θ, ω )}} .
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
5. Leveraged SV Model
Omori and Watanabe (2008) proposed a leveraged SV model with
normal errors (LSV-N model, hereafter) formulated as
1
Rt = σr e 2 ht ǫt ,
t = 1, . . . , T
ht+1 = φht+ σh vt ,
t = 1, . . . , T − 1
σh2
.
h1 ∼ N 0, (1−φ2
0
1 ρ
ǫt
∼ N
,
vt
0
ρ 1
They estimate both the parameter φ and parameter vector (σr , σh , ρ)
using MH algorithm separately and the latent volatility ht using multimove MH algorithm.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
6. Comparison on Real Data
The LSV model is fitted to the daily Japanese stock data used by Omori
and Watanabe (2008), that is TOPIX from August 1, 1997 to July 31,
2002.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
6. Comparison on Real Data (Cont’ed)
Tabel: Tuning parameters for the HMC and RMHMC implementations.
Sampler
HMC
RMHMC
Parameter of
sampler
h
NL
∆τ
NL
∆τ
NFPI
100
0.01
50
0.1
-
Parameter of model
φ
(σr , σh , ρ)
100
0.0125
6
0.5
5
Note: NFPI denotes the number of fixed point iterations.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
100
0.0125
6
0.5
5
6. Comparison on Real Data (Cont’ed)
Tabel: Posterior summary statistics.
Parameter
Mean (SD)
95% Interval
Panel A: Using MM-MH sampler
σr
1.259 (0.070) [1.121, 1.398]
φ
0.945 (0.019) [0.902, 0.974]
σh
0.193 (0.033) [0.138, 0.267]
ρ
−0.442 (0.103) [−0.630, −0.231]
Panel B: Using HMC sampler
σr
1.241 (0.070) [1.098, 1.377]
φ
0.954 (0.014) [0.925, 0.980]
σh
0.169 (0.025) [0.120, 0.219]
ρ
−0.461 (0.097) [−0.648, −0.272]
Panel C: Using RMHMC sampler
σr
1.238 (0.070) [1.098, 1.378]
φ
0.953 (0.014) [0.923, 0.980]
σh
0.171 (0.026) [0.122, 0.222]
ρ
−0.456 (0.098) [−0.636, −0.259]
D.B. Nugroho & T. Morimoto
IACT
Time (sec)
20.8
118.2
206.7
92.7
−
53.0
132.7
279.9
83.9
465.88
19.4
109.4
185.0
69.8
325.41
Comparison of MCMC Samplers
6. Comparison on Real Data (Cont’ed)
Tabel: Acceptance rates.
Parameter
h
φ
(σr , σh , ρ)
Acceptance rates
HMC
RMHMC
MM-MH
0.856
0.889
0.975
0.955
0.910
0.901
0.990
0.952
0.966
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
7. Conclusion
Based on the empirical results, we conclude:
The RMHMC sampler give the best performance in terms of autocorrelation time.
In particular, the HMC sampler exhibits slightly slower convergence
than MM-HMC sampler except for the leverage parameter.
Regarding the estimates of parameters, all samplers give similar estimates.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
Some References
Girolami, M. and Calderhead, B. (2011). Riemann manifold Langevin and
Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society, Series
B, 73 (2), 1–37.
Jacquier, E. and Polson, N. G. (2011). Bayesian methods in finance. In J.
Geweke, G. Koop, and H. van Dijk (Eds.), Handbook of Bayesian Econometrics.
Oxford University Press.
Kim, S. and Shephard, N. and Chib, S. (1998). Stochastic volatility: likelihood
inference and comparison with ARCH models. In N. Shephard (Ed.), Stochastic
Volatility: Selected Readings. Oxford University Press.
Omori, Y. and Watanabe, T. (2008). Block sampler and posterior mode estimation for asymmetric stochastic volatility models. Computational Statistics and
Data Analysis, 52, 2892–2910.
Takaishi, T. (2009). Bayesian Inference of stochastic volatility by Hybrid Monte
Carlo. Journal of Circuits, Systems, and Computers, 18 (8), 1381–1396.
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers
Thanks for your attention!
D.B. Nugroho & T. Morimoto
Comparison of MCMC Samplers