Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji 073500103288619287

Journal of Business & Economic Statistics

ISSN: 0735-0015 (Print) 1537-2707 (Online) Journal homepage: http://www.tandfonline.com/loi/ubes20

Estimation of Dynamic Bivariate Mixture Models
Roman Liesenfeld & Jean-François Richard
To cite this article: Roman Liesenfeld & Jean-François Richard (2003) Estimation of Dynamic
Bivariate Mixture Models, Journal of Business & Economic Statistics, 21:4, 570-576, DOI:
10.1198/073500103288619287
To link to this article: http://dx.doi.org/10.1198/073500103288619287

View supplementary material

Published online: 01 Jan 2012.

Submit your article to this journal

Article views: 45

View related articles

Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=ubes20
Download by: [Universitas Maritim Raja Ali Haji]

Date: 13 January 2016, At: 01:07

Estimation of Dynamic Bivariate Mixture Models:
Comments on Watanabe (2000)
Roman L IESENFELD
Department of Economics, Eberhard-Karls-Universität, Tübingen, Germany (roman.liesenfeld@uni-tuebingen.de)

Jean-François R ICHARD

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

Department of Economics, University of Pittsburgh, Pittsburgh, PA (fantinC@pitt.edu )
This note compares a Bayesian Markov chain Monte Carlo approach implemented by Watanabe with
a maximum likelihood ML approach based on an efŽcient importance sampling procedure to estimate
dynamic bivariate mixture models. In these models, stock price volatility and trading volume are jointly
directed by the unobservable number of price-relevant information arrivals, which is speciŽed as a serially

correlated random variable. It is shown that the efŽcient importance sampling technique is extremely
accurate and that it produces results that differ signiŽcantly from those reported by Watanabe.
KEY WORDS: Bayesian posterior means; EfŽcient importance sampling; Latent variable; Markov
chain Monte Carlo; Maximum likelihood.

1.

INTRODUCTION

Watanabe (2000) performed a Bayesian analysis of dynamic
bivariate mixture models for the Nikkei 225 stock index futures. Under these models, which were introduced by Tauchen
and Pitts (1983) and modiŽed by Andersen (1996), stock price
volatility and trading volume are jointly determined by the
unobservable ow of price-relevant information arrivals. In
particular, the high persistence in stock price volatility typically found under autoregressive conditional heteroscedastic
(ARCH) and stochastic volatility (SV) models is accounted for
by serial correlation of the latent information arrival process.
But because this process enters the models nonlinearly, the
likelihood function and Bayesian posterior densities depend
on high-dimensional interdependent integrals, whose evaluation requires the application of Monte Carlo (MC) integration

techniques.
Watanabe (2000) used a Markov chain Monte Carlo (MCMC)
integration technique, initially proposed by Jacquier, Polson,
and Rossi (1994) in the context of univariate SV models.
SpeciŽcally, he applied a variant of the Metropolis–Hasting
acceptance-rejection algorithm proposed by Tierney (1994)
(see also Chib and Greenberg 1995 for details) incorporating
a multimove sampler proposed by Shephard and Pitt (1997).
Details of this nontrivial algorithm were provided in appendix A of Watanabe’s article. Although Watanabe (2000) did
not report computing times, the MCMC algorithm is computerintensive (he used 18,000 draws for each integral evaluation),
and assessing its convergence is delicate in the presence of
highly correlated variables. Note, in particular, that his results
pass a convergence diagnostics (CD) test proposed by Geweke
(1992).
Watanabe’s Bayesian results for the bivariate mixture models are distinctly at odds with results found in the literature
for other datasets. SpeciŽcally, his estimates of volatility persistence are very close to those found for returns alone under univariate models. In contrast, the generalized method
of moments (GMM) estimates obtained by Andersen (1996)
for the U.S. stock market unequivocally indicate that persistence drops signiŽcantly under bivariate speciŽcations. This

Žnding is conŽrmed by the study of Liesenfeld (1998) for

German stocks. Liesenfeld (1998) computed maximum likelihood (ML) estimates using the accelerated Gaussian importance sampling (AGIS) MC integration technique proposed
by Danielsson and Richard (1993). This technique has since
been generalized into an efŽcient importance sampling (EIS)
procedure by Richard and Zhang (1996, 1997) and Richard
(1998).
In this present article, we rely on the EIS procedure to reestimate the Tauchen–Pitts model using Watanabe’s data. Our
ML estimates and posterior means all differ signiŽcantly from
Watanabe’s results and are fully consistent with the earlier Žndings of Andersen (1996) and Liesenfeld (1998). We show that
EIS likelihood estimates are numerically accurate even though
they are based on as few as 50 MC draws. We also demonstrate that Watanabe’s results are the consequence of an apparent lack of convergence of the implemented MCMC algorithm
in a single dimension of the parameter space. After discussions
between the editors, Shephard, Watanabe, and ourselves, it now
appears that the problem with Watanabe’s implementation of
the multimove sampler originates from a typographical error in
the work of Shephard and Pitt (1997). (A corrigendum to this article is soon to appear in Biometrika.) In the meantime, Watanabe has also corrected his own implementation and recently informed us that he now obtains results that are very similar to
ours (see Watanabe and Omori 2002 for further details). Hence
the focus of this article is to illustrate how EIS contributed to
detecting a problem of an MCMC implementation that passed
standard convergence tests. But EIS is more than just a procedure to check MCMC results—it is a powerful MC integration
technique on its own that should be considered as a potential

alternative to MCMC.
The remainder of the article is organized as follows. Section 2 briey reviews the bivariate mixture model of Tauchen

570

© 2003 American Statistical Association
Journal of Business & Economic Statistics
October 2003, Vol. 21, No. 4
DOI 10.1198/073500103288619287

Liesenfeld and Richard: Estimating Dynamic Bivariate Mixture Models

and Pitts (1983), and Section 3 outlines the EIS procedure. Section 4 compares the MCMC results of Watanabe (2000) for
the Tauchen–Pitts model with those obtained by EIS. Tentative explanations for the observed differences in these results
are discussed. Section 5 summarizes our Žndings and concludes.

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

2.

THE TAUCHEN–PITTS MODEL

Rt jIt » N.0; ¾r2 It /;

(1)

Vt jIt » N.¹v It ; ¾v2 It /;

(2)

and
´t » N.0; ¾´2 /;

3.

EFFICIENT IMPORTANCE SAMPLING

The objective of this section is to provide a heuristic description of EIS operation.Full technical and applicationdetails have
been provided by Richard (1998) and Liesenfeld and Richard
(2002). The integral in (4) can be written as

Z Y
T
f .Yt ; Xt jXt¡1 ; µ/ dX
L.µI Y/ D
(5)
tD1

Watanabe (2000) analyzed both the bivariate model of
Tauchen and Pitts (1983) and a modiŽed version thereof, proposed by Andersen (1996). Because the salient features of both
models are similar, here we discuss only the Tauchen and Pitts
version, which is given by

ln.It / D Á ln.It¡1 / C ´t ;

571

(3)

where Rt , Vt , and I t .t : 1 ! T/ denote daily return, trading
volume, and unobservable number of information arrivals. The

´t ’s are innovations, iid normally distributed. The model implies that, conditionally on It , Rt and Vt are normally distributed, independentlyfrom one another. Note that the conditional
variance of returns as well as the conditional mean and variance of volume are jointly driven by the common latent factor It . Furthermore, the marginal distribution of Rt jIt under (1)
and (3) coincides with that implied by the standard univariate
SV model. It is important to note that under the bivariate model
(1)–(3), the volume series provides sample information on the
latent process It , and thus on the dynamics of the return process
itself.
Because the latent variable It is serially dependent, the likelihood for the model (1)–(3) is given by a T-dimensional integral,
Z
L.µI Y/ D f .Y; Xjµ / dX;
(4)
where Y D fYt gTtD1 and X D fXt gTtD1 denote the matrix of the
observable variables Yt D .Rt ; Vt / and the vector of the latent variable Xt D ln.It /. The integrand represents the joint
density of Y and X, indexed by the unknown parameter vector µ D .¾r ; Á; ¾´ ; ¹v ; ¾v /0 . No closed-form solutions exist
for the likelihood (4), nor can standard numerical integration
procedures be applied. To estimate the dynamic Tauchen–
Pitts model, Watanabe (2000) applied a MCMC technique
that cycles between the distributions of µjXt ; Yt and Xt jµ; Yt .
Under the noninformative prior density used by Watanabe
(2000), the conditional distribution of µjXt ; Yt is obtained by

direct application of Bayes’s theorem, and sampling from it
is straightforward. In contrast, sampling from the distribution
of Xt jµ; Yt requires a rather sophisticated application of the
acceptance-rejection technique, as described in Watanabe’s appendix A.

and
f .Yt ; Xt jXt¡1 ; µ / D g.Yt jXt ; µ/p.Xt jXt¡1 ; µ/;

(6)

where g.¢/ denotes the bivariate normal density associated with
(1) and (2), and p.¢/ denotes the density of the AR(1) process
in (3). (For simulation purposes, one also must specify a distribution for the initial condition X0 ; this distinction is omitted
here for ease of notation.) A simple MC estimate of L.µ I Y/ is
then given by
#
" T
N
²
1 X Y ± e.i/

b
g Yt j Xt .µ/; µ ;
LN .µI Y/ D
(7)
N
iD1

tD1

Xt.i/.µ/gTtD1 denotes a trajectory drawn from the sewhere fe
.i/
Xt .µ/ is drawn from the
quence of p densities. SpeciŽcally, e
.i/
Xt¡1 .µ/; µ/. However, this simple proconditional density p.Xt je
cedure completely ignores the fact that observation of the Yt ’s
conveys critical information on the underlying latent process.
Therefore, the probability that a MC trajectory drawn from the
p process alone bears any resemblance to the actual (unobserved) sequence of Xt ’s is 0 for all practical purposes. It follows that the MC estimator in (7) is hopelessly inefŽcient. [As
documented by, e.g., Danielsson and Richard (1993), the range

of values drawn for the term between brackets in Equation (7)
often exceeds the computer oating point arithmetic capabilities.]
An EIS implementation starts with the selection of a sequence of auxiliary samplers fm.Xt jXt¡1 ; at /gTtD1 , typically deŽned as a straightforward parametric extension of the initial
samplers fp.Xt jXt¡1 ; µ/gTtD1 meant to capture sample information conveyed by the Yt ’s. For any choice of the auxiliary parameters at , the integral in (5) is rewritten as
L.µ I Y/
¶ T
Z Y
T µ
f .Yt ; Xt jXt¡1 ; µ/ Y
D
m.Xt jXt¡1 ; at / dX;
¢
m.Xt jXt¡1 ; at /
tD1

(8)

tD1

and the corresponding importance sampling (IS) MC likelihood
estimate is given by
e
LN .µI Y; a/

( T " ¡
¢ #)
.i/
N
Xt.i/ .at /je
Xt¡1
.at¡1 /; µ
1 X Y f Yt ; e
;
D
¡ .i/
¢
.i/
N
me
Xt .at /je
X .at¡1 /; at
iD1

.i/

tD1

(9)

t¡1

Xt .at /gTtD1 denotes a trajectory drawn from the sewhere fe
quence of auxiliary (importance) samplers m.
EIS then aims to select values for the at ’s that minimize the
MC sampling variance of the IS-MC likelihood estimate. This

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

572

Journal of Business & Economic Statistics, October 2003

requires achieving a good match between numerator and denominator in (9). Obviously, such a large-dimensional minimization problem must be broken down into manageable subproblems. Note, however, that for any given value of Yt and µ ,
the integral of f .Yt ; Xt jXt¡1 ; µ/ with respect to Xt does depend
on Xt¡1 (which actually represents the density of Yt jXt¡1 µ and
is generally intractable), whereas that of m.Xt jXt¡1 ; at / equals 1
by deŽnition. Therefore, it is not possible to secure a good
match between the f ’s and the m’s term by term.
Instead, EIS requires constructing a (positive) functional
approximation k.Xt ; Xt¡1 I at / for the density f .Yt ; Xt jXt¡1 ; µ/
for any given .Yt ; µ/, with the requirement that it be analytically integrable with respect to Xt . In Bayesian terminology,
k.Xt ; Xt¡1 I at / serves as a density kernel for m.Xt jXt¡1 ; at /,
which is then given by
m.Xt jXt¡1 ; at / D

1
k.Xt ; Xt¡1 I at /;
 .Xt¡1 I at /

with
 .Xt¡1 I at / D

Z

k.Xt ; Xt¡1 I at / dXt :

(10)

(11)

Note that a good match between f .Yt ; Xt jXt¡1 ; µ / alone and
k.Xt ; Xt¡1 I at / would leave Â.Xt¡1 I at / unaccounted for. But
because Â.Xt¡1 I at / does not depend on Xt it can be transferred
back into the period t ¡ 1 minimization subproblem. All in all,
EIS requires solving a simple back-recursive sequence of lowdimensional least squares problems of the form
aO t .µ/ D arg min
at

N
X
© £ ¡
¢
.i/
.i/
Xt .µ/je
Xt¡1 .µ/; µ
ln f Yt ; e
iD1

¢¤
¡ .i/
Xt .µ/I aO tC1 .µ/
¢Â e
¢ª2
¡ .i/
.i/
Xt¡1 .µ /I a t
Xt .µ/; e
¡ ° ¡ ln k e
(12)

for t : T ! 1, with Â.XT I aTC1 / ´ 1. [A weighted least squares
version of (12) can be found in Richard 1998.] As in (7),
.i/
fe
Xt .µ/gTtD1 denotes a trajectory drawn from the sampler p. (Although p is hopelessly inefŽcient for likelihood MC estimation,
it generally sufŽces to produce a vastly improved IS sampler on
a term by term least squares solution; a second and occasionally third iteration of the EIS algorithm, where p is replaced by
the previous stage IS sampler, sufŽces to produce a maximally
efŽcient IS sampler.) The Žnal EIS estimate of the likelihood
function is obtained by substituting fOat .µ /gTtD1 for fat gTtD1 in (9).
[For implementation of the EIS procedure for the bivariate mixture model (1)–(3), see the Appendix.]
It turns out that EIS provides an exceptionally powerful technique for the likelihood evaluation of (very) high-dimensional
latent variables models. Our experience is that for such models,
the integrand in (5) is a well-behaved function of X given Y and
thus can be approximated with great accuracy by an importance
sampler of the form given earlier, where the number of auxiliary
parameters in fat gTtD1 is proportional to the sample size. Likelihood functions for datasets consisting of several thousands observations of daily Žnancial series have been accurately estimated with MC sample sizes no larger than N D 50 (see, e.g.,
Liesenfeld and Jung 2000). In this article we use N D 50 MC

draws and 3 EIS iterations. Each EIS likelihood evaluation requires less than 1 second of computing time on a 733-MHz Pentium III PC for a code written in GAUSS; a full ML estimation,
requiring approximately 25 BFGS iterations, takes of the order
of 3 minutes.
In addition to its high accuracy, EIS offers a number of other
key computational advantages:
1. Once the baseline algorithm in (12) has been programmed
(which requires some attention in view of its recursive
structure) changes in the model being analyzed require
only minor modiŽcations of the code, generally only a few
lines associated with the deŽnition of the functions entering (12). For example, it took minimal adjustments in the
program for the estimation of the univariate SV model (1)
and (3) to obtain a corresponding code for the bivariate
mixture model (1)–(3).
2. High accuracy in pointwise estimation of a likelihood
function does not sufŽce to guarantee the convergence
of a numerical optimizer, especially one using numerical
gradients. Additional smoothing requires the application
of the technique known as that of common random num.i/
Xt .µ/gTtD1 draws for different
bers (CRNs), whereby all fe
values of µ are obtained by transformation of a common
et.i/ gT , typically uniset of canonical random numbers fU
tD1
forms or standardized normals. Note that such smoothing
is also indispensable to secure the convergence of simulated MC estimates for a Žxed MC sample size (see, e.g.,
McFadden 1989; Pakes and Pollard 1989). This particular issue is relevant when comparing EIS with MCMC.
Whereas CRNs are trivially implementable in the context
of EIS, in view of the typically simple form of the EIS
samplers, it does not appear feasible to run MCMC under
CRNs. Although this may not be critical for the computation of Bayesian posterior moments, which requires integration of the likelihood function rather than maximization, it does not appear feasible to numerically maximize
a likelihood function estimated by MCMC at any level of
generality.
3. Finally, because it takes only a few minutes to compute
ML estimates, it is trivial to rerun the complete ML-EIS
estimation under repeated draws of YjX; µ and XjY; µ for
any value of µ of interest (e.g., MC estimates or posterior means). Such reruns enable easy and cheap computation of Žnite-sample estimates of the numerical (XjY ),
as well as statistical (YjX) accuracy of our parameter estimates (Bayesian or classical) (see Richard 1998 for additional discussion and details). In the context of this article
where, as discussed in Section 4, EIS produces results that
are signiŽcantly different from those reported by Watanabe (2000), these reruns are an essential component of our
conclusion that there is a problem with Watanabe’s implementation of the MCMC algorithm.
4.

THE RESULTS

We estimated the SV model (1) and (3) as well as the dynamic Tauchen–Pitts model (1)–(3) with the ML-EIS procedure
for Watanabe’s data. The sample period starts on February 14,

Liesenfeld and Richard: Estimating Dynamic Bivariate Mixture Models

573

Table 1. MCMC and ML-EIS Estimation Results for the SV and Tauchen–Pitts Models

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

¾r

Á

¾´

SV Model
MCMC
1:053
.:084/
[:0038]
ML-EIS
1:156
.:098/
[:0005]

:968
.:013/
[:0004]
:965
.:017/
[:0003]

:156
.:021/
[:0008]
:174
.:041/
[:0009]

Tauchen–Pitts Model
MCMC
1:247
.:042/
[:0036]
ML-EIS
1:180
.:035/
[:000006]

:974
.:008/
[:0002]
:801
.:029/
[:000040]

:064
.:003/
[:0001]
:217
.:013/
[:000024]

¹v

¾v

Log-likelihood

¡1;418:4
[.0793]
2:819
.:139/
[:0169]
2:621
.:092/
[:000005]

:669
.:029/
[:0021]
:380
.:034/
[:000060]

¡2;435:0
[.0124]

NOTE: Asymptotic (statistical) standard errors are in parentheses, and MC (numerical) standard errors are in brackets. The MCMC
posterior means and the MC standard errors of the posterior means are obtained from table 3 of Watanabe (2000). The MCMC
posterior standard deviations are calculated from Watanbe’s draws from the posterior distribution. The ML-EIS estimates are based on
a MC sample size N D 50 and three EIS iterations, and its asymptotic standard errors are obtained from a numerical approximation to
the Hessian.

1994 and ends on October 1, 1997, giving a sample of 899 observations. Whereas Watanabe (2000) computed Bayesian posterior means, we initially computed ML estimates. As we illustrate, the actual differences between ML estimates and posterior means are very small, unsurprisingly in view of the facts
that the likelihood functions of the models that we are considering are very well-behaved, and that Watanabe (2000) used a
noninformativeprior. His MCMC posterior means and standard
deviations together with our ML-EIS estimates, are reported in
Table 1.
For the SV model, the MCMC and ML-EIS results are very
similar. In particular, the persistence parameter Á is estimated at
.97 under both methods. Furthermore, both procedures exhibit
similar degrees of numerical accuracy. This can be seen from a
comparison of the MC (numerical) standard errors (for the MLEIS estimates, they are obtained by Žtting the model 20 times
under different sets of CRNs and computing the corresponding
standard deviations).
In contrast, the divergence between the ML-EIS and Watanabe results is striking for the Tauchen–Pitts model. In particular, the ML-EIS method produces signiŽcantly smaller estimates of the persistence parameter, Á, and the variance parameter for volume, ¾v . In addition, the ML-EIS estimate for the
variance parameter of the information arrival process, ¾´ , is
substantially larger than that reported by Watanabe (2000). It is
important to note that the Žnding that the ML-EIS estimate of
Á drops signiŽcantly (from .97 to .80) for the bivariate model is
fully consistent with the GMM results of Andersen (1996) for a
modiŽed version of the Tauchen–Pitts model estimated for U.S.
stocks and with the ML-AGIS results of Liesenfeld (1998) for
the Tauchen–Pitts speciŽcation for German stocks. In contrast,
Watanabe’s MCMC posterior means of Á are nearly identical
under both speciŽcations. Finally, it can be seen that the MC
(numerical) standard errors for the ML-EIS estimates of the
Tauchen–Pitts model are much smaller than those obtained for
the univariate SV model. This makes perfect sense, because the
EIS procedure applied to the bivariate model exploits the additional information provided by the series of trading volumes on
the behavior of the latent process.

Table 2 displays the estimated asymptotic correlation matrix
for the ML-EIS estimates as obtained from a numerical approximation to the Hessian. Interestingly,the signs of the differences
between the ML-EIS and Watanabe’s parameter estimates are
the same as those of the corresponding correlations. For example, the ML-EIS estimator of Á is positively correlated with
that of ¾v and negatively correlated with that of ¾´ . This is fully
consistent with the fact that a higher estimate of Á by Watanabe
coincides with a higher estimate of ¾v and a lower estimate of
¾´ relative to their corresponding ML-EIS counterparts.
First, we considered the possibility that be Žnite-sample biases may be inherent in the ML-EIS estimation procedure. To
address this concern, we conducted two simulation experiments
in which we draw 100 Žctitious samples of size 899 from the
Tauchen–Pitts model. In the Žrst experiment, the parameters
were set equal to their ML-EIS estimates; in the second experiment, at their MCMC posterior means (as given in Table 1).
ML-EIS estimates were then computed for each simulated sample. MC means and standard deviations of the ML-EIS estimates under each set of parameter values are reported in Table 3. It is quite obvious that the ML-EIS estimation procedure
performs very well under both sets of parameter values. Furthermore, it is worth noting that the Žnite-sample standard errors obtained under the ML-EIS parameter values are in very
close agreement with the asymptotic standard errors of these
estimates as given in Table 1.
We next considered the possibility that the log-likelihood
function of the Tauchen–Pitts model might be at for this particular dataset. Hence we Žrst computed the log-likelihood
Table 2. Estimated Correlation Matrix for the ML-EIS Estimates of the
Tauchen–Pitts Model

¾r
Á
¾´
¹v
¾v
NOTE:

¾r

Á

¾´

¹v

1:000

:076
1:000

¡:094
¡:632
1:000

:594
:074
¡:114
1:000

The statistics are obtained from a numerical approximation to the Hessian.

¾v
:197
:598
¡:776
:264
1:000

574

Journal of Business & Economic Statistics, October 2003

Table 3. Mean and Standard Error of the ML-EIS Estimator for the
Tauchen–Pitts Model

True value
Mean
Standard error
True value
Mean
Standard error

¾r

Á

¾´

¹v

1:180
1:183
:030
1:247
1:249
:048

:801
:794
:029
:974
:968
:011

:217
:217
:014
:064
:065
:007

2:621
2:622
:087
2:819
2:811
:202

¾v
:380
:376
:035
:669
:666
:030

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

NOTE: The statistics are based on 100 simulated samples, each consisting of a time series
of length 899. The ML-EIS estimates are based on a MC sample size N D 50 and three EIS
iterations.

function at Watanabe’s posterior means and at our ML-EIS estimates. The correspondingvalues equal ¡2;572:1 and ¡2:435:0,
implying a highly signiŽcant log-likelihooddifference of 137.1,
immediately ruling out the possibility of a at log-likelihood
function. This conclusion is supported by Figure 1, which
shows sectional plots of the log-likelihood function at the MLEIS parameter values. Obviously,the log-likelihoodis very well
behaved around these estimates. Figure 2 displays similar plots
at Watanabe’s posterior means. Note that whereas Watanabe’s
posterior means are very far from our ML-EIS estimates, his
posterior means of (¾r ; Á; ¹v ; ¾v ) are close to maximizing a
log-likelihood function in which ¾´ is set equal to its posterior
mean. In other words, Watanabe’s posterior means and the ML-

Figure 2. Sectional Log-Likelihood Functions for the Tauchen–Pitts
Model at Watanabe’s MCMC Posterior Means of the Parameters: (a) ¾r ;
(b) ¹v ; (c) ¾v ; (d) Á ; (e) ¾´ .

EIS estimates of these four parameters are fully consistent with
one another, conditionallyon ¾´ . This does suggest that Watanabe’s initial algorithm might have got stuck in the ¾´ dimension
but converged in the four remaining dimensions.
Finally, to verify that ML estimates and posterior means
are very close to one another for the well-behaved dataset,
we calculated the posterior means based on the priors used by
Watanabe using an IS technique. The posterior mean of the pa0
Rrameter vector µ DR.¾r ; Á; ¾´ ; ¹v ; ¾v / is given by E.µjY/ D
µ L.µ I Y/p.µ/ dµ= L.µI Y /p.µ/ dµ . Hence, compared with
the likelihood function, the integrals in the numerator and denominator of E.µ jY/ exhibit Žve additional dimensions, one for
each parameter. Watanabe adopted the following noninformative priors for the elements of µ :
p.¹v / D I[0; 1];
p.¾r / / 1=¾r ;

Figure 1. Sectional Log-Likelihood Functions for the Tauchen–Pitts
Model at the ML-EIS Estimates of the Parameters: (a) ¾r ; (b) ¹v ; (c) ¾v ;
(d) Á ; (e) ¾´ .

p.Á/ D I[¡1; 1];

p.¾v / / 1=¾v ;

p.¾´ / / 1=¾´ ;

where I[a; b] is the indicator function of the interval [a; b]. If
m.µ/ is an IS density for µ , then an importance MC estimate of
E.µ jY/ can be obtained as
£ .j/
¯
¤
1 PJ
µQ e
LN .µQ .j/ I Y/p.µQ .j// m.µQ .j/ /
jD1
J
b
E.µ jY/ D 1 PJ £
¯
¤ ;
e Q .j/
Q .j/
Q .j/
jD1 LN .µ I Y/p.µ / m.µ /
J

Liesenfeld and Richard: Estimating Dynamic Bivariate Mixture Models
Table 4. Posterior Means Evaluated by Importance Sampling

Posterior mean
MC standard error

¾r

Á

¾´

¹v

¾v

1:177
:034

:816
:030

:220
:018

2:628
:117

.354
.032

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

NOTE: The posterior means are computed by using a MC sample size of 100, and the MC
standard errors are obtained from 20 evaluations of the posterior means each based on a MC
sample size of 100. The EIS evaluations of the likelihood are based on a MC sample size N D 50
and three EIS iterations.

where fµQ .j/ gJjD1 is a simulated sample of J iid draws from m.µ /
LN .µQ .j/I Y/ is the value of the likelihood function at draw
and e
Qµ .j/ obtained by the EIS procedure. An inspection of the sectional likelihood functions at the ML-EIS estimates of µ (not
presented here) revealed that they have shapes very similar to
Gaussian densities. Hence we simply used a multivariate normal distribution as importance sampler m.µ / with mean and
variance given by the ML-EIS estimate of µ and the corresponding estimated asymptotic variance-covariance matrix. (Had we
found that the posterior densities were less well behaved, we
might have reprogrammed our EIS algorithm to include these
Žve additional dimensions, but it was clearly not justiŽed here.)
The IS MC estimates of the posterior means based on an MC
sample size of J D 100, together with their MC standard errors, are reported in Table 4. Even though these estimates have
relatively large MC standard errors (relative to the EIS standards), they are sufŽciently accurate to conclude that all posterior means are very close to the ML-EIS estimates given in
Table 1. All in all, we Žnd that the posterior densities of all Žve
parameters are very well behaved for this dataset and cannot
provide a rationale for the failure of Watanabe’s MCMC algorithm to converge in the ¾´ dimension.
5.

CONCLUSIONS

We have estimated a dynamic bivariate mixture model with
a ML procedure based on an EIS technique and obtained parameter estimates that differ signiŽcantly from those reported
by Watanabe (2000), who used a Bayesian MCMC multimove
procedure for the same model and the same data set. Although
our ML-EIS estimates are in accordance with the Žndings reported in the literature, Watanabe’s are at odds with those. In
particular, his estimate of the persistence parameter of the latent
information arrival process, which drives the joint dynamics
of trading volume and return volatility, is signiŽcantly higher
than the corresponding ML-EIS estimates, whereas his estimate of the variance parameter of the same process is much
lower. Exploring possible explanations for these differences,
we found that the ML estimates based on EIS are numerically highly accurate. Moreover, it appears that although Watanabe’s MCMC implementation fails to converge in the dimension of the variance parameter of the latent information arrival
process, his estimates of the four remaining parameters can be
explained by the ML-EIS procedure. Because the observed differences in the estimates of the persistence and variance parameter of the information arrival process are critical for an
ex post analysis and forecasts of return volatility, we need to
be particularly concerned about the possibility of implementation errors. Whereas MCMC has repeatedly demonstrated its
numerical capabilities across a wide range of applications, this

575

article ought to serve as a warning that MCMC implementation problems might not be easily detectable. Note in particular that Watanabe’s results did pass standard convergence tests.
Obviously, the fact that we now have an alternative and powerful algorithm with EIS will help on this front. But EIS does
more than just verify MCMC—it has repeatedly demonstrated
high numerical accuracy, ease of implementation, and numerical stability. Also, its computational speed allows for running
full simulation experiments to produce Žnite-sample measures
of both numerical and statistical accuracy. Running such simulation experiments with large sample sizes also provides an
operational veriŽcation of the validity of any particular EIS
implementation. In conclusion, EIS is deŽnitely an algorithm
worthy of consideration on his exceptional numerical performance.
ACKNOWLEDGMENTS
The authors thank Toshiaki Watanabe for his very helpful and
constructive cooperation, and former editor Jeffrey Wooldridge
and an associate editor for their helpful comments.
APPENDIX: IMPLEMENTATION OF EFFICIENT
IMPORTANCE SAMPLING FOR THE
TAUCHEN–PITTS MODEL
Computation of an EIS estimate of the likelihood function
for the bivariate mixture model (1)–(3) for a given value of the
parameter vector µ requires the following steps:
Step (0). Draw a set of N £ T independentrandom numbers
.i/
ff e
Ut gTtD1 gN
iD1 from a standardized normal distribution. These
draws are used as CRNs during the complete EIS procedure
to generate trajectories of the latent variable. Then use the initial samplers p to draw N trajectories of the latent variable
fe
Xt.i/ .µ/gTtD1 . According to (3) and (6), the initial samplers are
characterized by the density of a Gaussian AR(1) process given
by p.Xt jXt¡1 ; µ / / expf¡.Xt ¡ ÁXt¡1 /2 =2¾´2 g, where multiplicative factors that do not depend on Xt are omitted.
Step ( t ); (T: ! 1). Use the random draws from the initial
samplers to solve the back-recursive sequence of the T least
squares problems deŽned in (12). The relevant functional forms
are obtained as follows. A natural choice for the auxiliary samplers m is to use parametric extensions of the initial samplers
p. This implies that the density kernel for the auxiliary sampler
m.Xt jXt¡1 ; at / be a Gaussian density kernel for Xt given Xt¡1 ,
which can be parameterized as
k.Xt ; Xt¡1 I at / D p.Xt jXt¡1 ; µ /³.Xt ; at /;

(A.1)

where ³ is an auxiliary function given by ³ .Xt ; at / D
expfa1;t Xt C a2;t Xt2 g, with at D .a1;t ; a2;t /. Note that under this
parameterization, the initial sampler p cancels out in the least
squares problem (12). Inserting the functional forms of p and ³
in (A.1) leads to the following form of the density kernels,
µ³
´
»
1 ÁXt¡1 2
k.Xt ; Xt¡1 I at / / exp ¡
¾´
2
´
³
´ ¶¼
³
ÁXt¡1
1
C a1;t Xt C
¡ 2a2;t Xt2 ; (A.2)
¡2
¾´2
¾´2

576

Journal of Business & Economic Statistics, October 2003

where the conditional mean and variance of Xt on m are
given by
Á
!
¾´2
ÁX
t¡1
2
¹t D ¾t2
: (A.3)
¾
D
C
a
and
1;
t
t
¾´2
1 ¡ 2¾´2 a2;t

Downloaded by [Universitas Maritim Raja Ali Haji] at 01:07 13 January 2016

Integrating k.Xt ; Xt¡1 I at / with respect to Xt leads to the following form of the integrating constant:
¼
» 2
¹t
.ÁXt¡1 /2
 .Xt¡1 I at / / exp
;
¡
(A.4)
2¾´2
2¾t2
where irrelevant multiplicative factors are omitted.
Based on these functionalforms, the step t least squares problem (12) is characterized by the linear auxiliary regression
¢
¡
¢
¡ .i/
.i/
Xt .µ /I aO tC1 .µ/
Xt .µ/; µ C ln  e
ln g Yt je
£ .i/ ¤2
.i/
Xt .µ /
Xt .µ/ C a2; t e
D constant C a1; te
C residual;

i : 1 ! N;

(A.5)

where g.¢/ is the bivariate normal density for Yt given Xt associated with (1) and (2). The initial condition for the integrating
constant is given by Â.XT ; ¢/ ´ 1.
Step (T C 1). Use the Gaussian samplers fm.Xt jXt¡1 ;
aO t .µ//gTtD1 , which are characterized by the conditionalmean and
.i/
Xt .Oat .µ//gTtD1 ,
variance given in (A.3), to draw N trajectories fe
from which the EIS estimate of the likelihood is calculated according to (9).
[Received August 2000. Revised November 2002.]

REFERENCES
Andersen, T. G. (1996), “Return Volatility and Trading Volume: An Information Flow Interpretation of Stochastic Volatility,” Journal of Finance, 51,
169–204.
Chib, S., and Greenberg, E. (1995), “Understanding the Metropolis–Hastings
Algorithm,” The American Statistican, 49, 327–335.

Danielsson, J., and Richard, J. F. (1993), “Accelerated Gaussian Importance
Sampler With Application to Dynamic Latent Variable Models,” Journal of
Applied Econometrics, 8, S153–S173.
Geweke, J. (1992), “Evaluating the Accuracy of Sampling-Based Approaches
to the Calculation of Posterior Moments,” in Bayesian Statistics Vol. 4, eds.
J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, Oxford, U.K.:
Oxford University Press, pp. 169–193.
Jacquier, E., Polson, N. G., and Rossi, P. E. (1994), “Bayesian Analysis of Stochastic Volatility Models,” Journal of Business & Economic Statistics, 12,
371–389.
Liesenfeld, R. (1998), “Dynamic Bivariate Mixture Models: Modeling the Behavior of Prices and Trading Volume,” Journal of Business & Economic Statistics, 16, 101–109.
Liesenfeld, R., and Jung, R. C. (2000), “Stochastic Volatility Models: Conditional Normality Versus Heavy-Tailed Distributions,” Journal of Applied
Econometrics, 15, 137–160.
Liesenfeld, R., and Richard, J. F. (2002), “Univariate and Multivariate Stochastic Volatility Models: Estimation and Diagnostics,” Journal of Empirical Finance, forthcoming.
McFadden, D. (1989), “A Method of Simulated Moments for Estimation of
Discrete Response Models Without Numerical Integration,” Econometrica,
57, 995–1026.
Pakes, A., and Pollard, D. (1989), “Simulation and the Asymptotics of Optimization Estimators,” Econometrica, 57, 1027–1058.
Richard, J. F. (1998), “EfŽcient High-Dimensional Monte Carlo Importance
Sampling,” unpublished manuscript, University of Pittsburgh, Dept. of Economics.
Richard, J. F., and Zhang, W. (1996), “Econometric Modeling of UK House
Prices Using Accelerated Importance Sampling,” The Oxford Bulletin of Economics and Statistics, 58, 601–613.
(1997), “Accelerated Monte Carlo Integration: An Application to Dynamic Latent Variable Models,” in Simulation-Based Inference in Econometrics: Methods and Application, eds. R. Mariano, M. Weeks, and T. Schuermann, Cambridge, U.K.: Cambridge University Press.
Shephard, N., and Pitt, M. (1997), “Likelihood Analysis of Non-Gaussian
Measurement Time Series,” Biometrika, 84, 653–668; corr. forthcoming.
(2003),x,xx.
Tauchen, G. E., and Pitts, M. (1983), “The Price Variability-Volume Relationship on Speculative Markets,” Econometrica, 51, 485–505.
Tierney, L. (1994), “Markov Chains for Exploring Posterior Distributions,” The
Annals of Statistics, 21, 1701–1762.
Watanabe, T. (2000), “Bayesian Analysis of Dynamic Bivariate Mixture Models: Can They Explain the Behavior of Returns and Trading Volume?,” Journal of Business & Economic Statistics, 18, 199–210.
Watanabe, T., and Omori, Y. (2002), “Multi-Move Sampler for Estimating NonGaussian Time Series Models: Comments on Shephard and Pitt (1997),”
unpublished manuscript, Tokyo Metropolitan University, Faculty of Economics.