BAYES ESTIMATION FOR ARMA MODEL FORECASTING UNDER NORMAL-GAMMAR PRIOR.
Bayes Estimation for ARMA Model Forecasting
under Normal-Gamma Prior
Zul Amryl'* and Adam Baharum2
1
2
Department of Mathematics, State University of Medan, lndonesia
School of Mathematical Sciences, Universiti Sains Malaysia, Pulau Pinang, Malaysia
*Corresponding
Author: zu l.a ryrrv@gmpil,cop
Presented
On The 3'd International Seminar on Operation Research.
To Celebrate The 50th Department of Mathematics University of
Sumatera Utara, Medan, lndonesia.
August 21-23,20L5.
Bayes Estimation for
ARMA Moder Forecasting under
Normal-Gamma Prior
Zul Amryl,- and Adam Baharum2
2schoo,",T:ffi:ll:1iT:*H',il1,::?:i,"Ji;Jil:,$H:fr1;i,Ti#x'ilMa'aysia
-Corresponding
Abstract
Author: zJl.am{r@gmail.com
This paper presents a Bayesian approach to find the Bayes estimator for the point forecast
and forecast variance
of
ARMA model under normal-gamma prior assumption with quadratic loss function in the form mathematical
expression. The
conditional posterior predictive density be obtained based the posterior under normal-gamma
prior and the conditional
predictive density' whereas the marginal conditional posterior predictive density
be obtain-ed based the conditional posterior
predictive density. Fufihermore, neither the point forecast nor the forecast variance
are both derived from the marginal
conditional posterior predictive density.
Keywords ARMA
1.
Model, Bayes Theorem, Normal-Gamma prior
Introduction
Bayes theorem calculates the posterior distribution as proportional to the product
of a prior distribution and the likelihood
function' The prior distribution is a probability model desiribing the knowledge about ihe parameters
before observing the
currently by the available data [1 ]. Main idea of Baye_sian forecaiting is the prrii.tiu"
distribution of the future given thJfast
data follows directly from the joint probabilistic model. The predictlve distiibution
is derived from the sampling predictive
density, weighted by the posterior dishibution
[4]. This pup"i ir done refer to Liu [15] discussed the Bayesian u"nutyiri, ro,
one-step ahead forecast in ARMA model and Liu
[16] also discussed the multiperiod ftrecast for ARX model employed the
Bayesian approach. others the paper related to this research are Amry and Baharum
[2],Fan& yao [7], Kleibergen & Hoek
[13]' and Uturbey [20] also discussed the Bayesian analysis for ARMA model. This'paper focuses to find the mathematical
expression of the Bayes estimator for the point forecast and forecast variance
of ARMI model under normal-gamma prior
assumption with quadratic loss function
2. Materials and Methods
The materials in this paper are some theories in mathematics and statistics such
as the ARMA model, Bayes theorem,
repeated integration, gamma distribution, and the univariate student's t-distribution.
The method is study of literature by
applying the Bayesian analysis under normal-gamma prior assumption.
2.1. ARMA model
The ARMA (1t, q)model [15] defined by:
y, =f.o y,-,
i=t
where {e'}
parameters.
is
sequence
of i i d
* j=tfe,r, , * r,
(2.1)
'
normal random variables with e,-N(0,rt1,
r>0
and unknown,
0,
and
Q
are
2.2.Bayes theorem
Suppose there are two discrete random variables
atd. )r, then the joint probability function can be written
p( x,y) = p( xl y) p,( y) and the marginal probability density function
of
[18] is:
p*(x)= z, p(x,y) 2,p(x|y) p,( y)
(2.2)
Bayes rule for the conditional p( y I x
is
X
:
)
X
p(
IfIis
ylx)= p(x,y) :
P*(r)
p,(y) _ p(xly) p,(y)
p,(x)
Z,p(xly)pr(y)
p(xly)
continuous, the Bayes theorem can be stated as:
,r
r'
(2.3)
r1=- PQ-2)-P'1! tj p(xly) p,r y )dy
(2.4)
by using the proportional notation (oc which can be expressed by:
)
p(ylx) 6 p(x)y)p,(y)
where p( y I x) is posterior distribution, p(x
I
I
(2.s)
is likelihood function and p,( y )is prior distribution.
2.3. Gamma distribution
A positive random quantity
/
is said to have a gamma distribution with paramet er n
>0
density function [17]:
e@=#ia*'
the notation isQ
- c(n,d)
the mean is
o(a)=]
and
d>0 if it has the probability
expf-4al
(2.6)
is var(Q)=ft
and the variance
2.4. Univariate student-t distribution
A random quantity, x, is said to have a student-t distribution on n degrees
of freedom with mode
r > 0if it has the probability density function [17]:
r( n*1),1
p1x
-
y) l)-1,*6- r
r
.,
r')'
rl;)@:
Thenotationisx-t,(p,r),tnemeanis
r(x)=p
-
p and scale parameter
,,,
|'
(2.7)
l
andthevariance
isvar{x)=J7-,ifn>2
3. Results
3.1. Likelihood Function for ARMA model
The k-step-ahead point forecast ofl ,* t , defined by
:
i(k)=E(Y,*ls]:)
(3.1)
where Sj =(y,, y,,..., y,*o_,)
Based the equation (2.
l)
be obtained residuals as:
e, = y,
-tt,y,-,
-f,o,r,_,
(3.2)
€,: e where r: min(0, p+l-q),
byBox&Jenkins[4],thelikelihoodfunctionfory:(Q1,Q2 ,000t,02,..,0)andr
based sj
By conditioning the first p observations and lerling eo:eo-t:...:
/ r s])x
-\
L\v,
I .1,-r ,f p
,
t,ll
r , p^rl-iLLlr,-Lo,t
-2t,,
one may approximate
t,
,n'k-t'
(3.3)
))j
The equation (3.3) can be expressed as:
/
\
r(p.rlsjJ
where B,
: (y,, !tt,
(n.k-ttp
(
cr , *fl-tllti
z
_1._o_,
I
..., lt+t_p, e6 er_t, ..., €*t_q)
I t=p+r
-
2y'
nrk-t
n+k-
|
ll
y @, a,_,)' ll
L /,8,_,+ t=p.r
t=p. r
J)
(3.4)
By
letting
Yp
!p+t
!n*-2
lp-t
Yp
!n+k,3
/ t
/z
/n+k-t-p
€o
€p*t
€n+k-2
p-t
€,
€n+k-j
U=
€
.'li';'i:l
W:U(.f,and V:Uxo
en+k-t-p
p*t-q € p*z-q
p _c_r
€
where 6,=y,-2d,y,,-ZF,aFj,t:p+l,p+2,...,n, 6,
a,, a,-,,..., a,-oo.
& d, aremaximumlikelihoodestimator of Q, & 0,,and
lutua.a iru'
A,
=y,
-Fr
B,_,
(3.5)
the likelihood function in equation (3.4) can be expressed as:
t\v.r
(n-k-tt-p
{
ll
|
lj
-f*r-r
rrpl-=llyl
2I,A;-zy'v+v'wvll
s))ne,
3.2. Posterior distribution under normal-gamma
(3.6)
prior
According to Broemeling and Shaarawy's suggestion [5], the normal-gamma prior of parameters Pand ris:
€(Y,r)=6,(Y )r).
n
(,(r)
r'f-'*o{-1lr'g*-v'ep-p'ev+pre1.t+2Bj}
'l 2'
,.tl
)
where f, - ,Ut,Gd-')
,
I , - c.tu (a, p) , Q is apositive definite matrix of the order (p + q),
By applying Bayes theorem to equation (3.6) and (3.7), the posterior
!o and r-, is:
of
n+k-t-p'2d+p_t
q.
p:w+e
and. K
=')t*,,ri
+ 1tr e1t +
2
p
andB are parameters.
1= r(v,r1s)), a(v,,)
n(Y,r-t1si
where
a
(
,t n
*pI-jlv,ev-v,
(v+ep)-(y+q1,fv+x)j
r,
(3.8)
.
3.3. Conditional posterior predictive density
Based
on
", = ,, -f,O,r,-,
-fr,",-,
with
e,
- u(o,r-,
)
be obtained f (et1s:,y,r-t )= (zo,
,)r *o{_LrG),}
Ifexpressed in y, is:
LI
f(y,lS),V,r-' 1=(2r, ' ''-t
)' uP\-jl,
P
-LQ,Y,-, -
Based the equation (3.9), be obtained the conditional predictive density
f
(
y.,o
I
s),y,
t -' ) =
(z
o,-' I i
*r{-
P,tr,,f' I
of y , *
(3.e)
p..
}lr. " - fit, r.. 0,, - f ,, n,-,)' }
p
o
l'l
*,'L*, f-f;Lt,.r _l,., t,-o-, _\t,,,*_,
)]
I_
* ': *r{-+l ,.-r -lf.r*.---, .ir,"..--,ll'l
"L
[
Bychanging f4,y,,o-,+f,o,r,*-, to:
\,=i
i-t
)))
(3.1 0)
!n+k-l
!
Qt!*t-t *02!n,*-z t .,.1-dp!,*-p
where
B*k-t=0*u,,
+ete,+k-t t0z€n+*-z +,..+eqe,+k_q
!n+k-2,..., !o+r-p,e,11-1,
-f
en+k-2,
6,.0 | s;, v,
-,
"
: (Ot Q, ,
0,
e
j e2
n+k-2
!n+*-p
0r)
€ n+k-t
€
n+k-z
€
n*-q
: V, B,*O-'
..., e,*o_o), theequation (3.10) can be written as:
*,i
*o
*
*p7-ilf,-r
)
,1
{-
!r[ r,. r -
r,
u,
* _,]']
- zv, a,-* t rn+t
* @, u*r-,)'lj
f ,r *v, Ry 2v7 t,,r_,
* ,iI *pl-ily',-o
,,-r lI
(3.1l)
where R = Bn+r_t@ BI*o_, and (v, n,*o_,)t =v, nv
Basedontheequation(3.8)and(3'll),beobtainedtheconditionalposteriorpredictivedensity
f o(t,*ls:,,Y,r-t
)
n o(v,
'"
ofy,t
lsj ) f (y,.ols),v)1 I
B,u-, y*o)- f)
,-o-,-o-r".0-, I lv'av -Yr 1lt
T'
*ofil(y +ep)r BI**., !,.* 1v * yl.r * rcll
+Q1t+
0.,
t"Lll
whereG:P+R
+
(3.12)
3.4. Marginal conditional posterior predictive density
The marginal conditional posterior predictive density
predictive density in equation (3.12):
_f
,(t,.rls,)
=
I[
_:-
=
-f ,f
of Y ,* r be obtained by integrating
the conditional posterior
t,,rls:,y,r1) dydr
ii''-*'-{-
;l:;,o.*r;'.' :;r':)l ;; i;-,:r' : -)}
n
*
o,
z--' ^'_l(r_o-,(1tt+g1t1+n,+k.trn+k)), c(v_c_,(1v+e1t)+8,*r,y,_r)\_f ,...,
* J71,--x-4:]:!_,
J'
o'
"-'l'i, )nu,* B,t*-t !n-t ), c-, (rt +gpt* B,-*-r
t,,* )*y,,,0 + K
)o'
(n+k-l- p+2d )+l
2
*l'.0-'-0.'"
*u -Q - Bl,o-,
G-t
B,*r-t
_K-{V+ep)t
+
(n+ k
- t - p+ 2")
U
f'
(t:,0-, c-, g, *
p1.,1)]
(3.1 3)
Gotv+gp1
-
B|*, B,;J
The marginal conditional posterior predictive density of y,*1
is a univariate student,s t_distribution on (n+k_l_p+2a)
degrees of freedom with mode
(t
a1-o-,
G-' Bn*o-,)" (u:.r-, c' 1v + g1t1) and scale parameter
tr= -
_
T=
(n + k
K-(V+eu)rG^(V+Ou)
v\
-"
- t- p+ za)Q -
ffi+;J'
where
co =G-t *Q
-
Bl.o-,G-'8,-r-,)-'
(c-'ao
'1
3.5. Point forecast and forecast variance
For quadratic loss function, the point forecast of Y
, *p is the posterior mean of the marginal conditional posterior predictive,
that is:
a(a., t s; )=(, -
Bf,*o_, G-,
B,*o_,)-' (r:.0_, c-' 1v + gp 1)
(3.14)
and the forecast variance
of
I,
* r is:
K-(V+
(n+k-1- p+2a)
r*(v,,.r 1s')=
1r
Golv +gpy
+k-l-p+2a
=(n + k
- 3-
p
+ za)-' (x - p,
+
gp 1, c o1v
+
gp 1)
Q
- ul,*,
G -,
B,*o_,),
(3. 1 5)
4. Conclusion and computational procedure
This paper analyzes how to find mathematical expression ofthe point forecast in equation (3.14)
and the forecast variance
in equation (3.15) based the marginal conditional posterior prediitive density in aquation
The conditional posterior
f:.r:1.
predictive density be obtained by multiplying the normal-gamma prior to the condiiional piedictive
density. By iniegrating
the conditional posterior predictive density to paramaters Y and t respectively, obtained
tire marginal .onditionut po-sterior
predictive density. Furthermore, the point forecast and the forecast uuiiun.. are derived
based the mean and the variance of
marginal conditional posterior predictive density that has the univariate student's t-distribution. procedure
to compute the
point forecast and the forecast variance are as follows:
Defines:
Yp
!p+t
!n+r-:
lpt
Yp
!n+k,3
!t
y.
,vh+x t_p
::::
U=
ao
6
€
o_,
iii.
iv.
XO:
€n+k-2
o
€ n+k-3
p*z-q
en+k-t-q
:::
e
ll.
p*l
€
€
l';:.:,1
l:l
lr..r
I
It, Q, a and
B
B,* -, =\y *o -,' !.nr-l,..., / n+t-t-p, € n+k,t,
€ n+*-2,
..., r,*-,u)
Compute:
v.
W:ULf
vi.
V:Ux6
P:IY+Q
^.f'
viii. K =
yl * p'ep + 2p
vii.
ix.
R= B,,r_,& BI*_,
x.
G:P+R
xi.
G-1
xii.
Go
xiii.
xiv.
= G-t + Q - Bl.o_,G-'a,,r_,)-'(c-'nc-')
E(y,.k s: )
)
:
Q
- a:.r-, G-' B,*0,,)-' (u:.,,, c-, 1v + gp 1)
va,(r.ul $) = (, + k - 3 - p + 2a)-'
(*
- f n + ep
f
G,(v + ep ))
Q
-
B!,,r_, G-,
B.*_,)-,
References
lll
Alba, E. And Mendoza, M. Bayesian Forecasting Methods for Short
Time Series. Article
l2l
Amry, Z' and Baharum, A. (2015)' Bayesian Multiperiod Forecasting
for Arma Model under Jeftey,s prior.
Mathematics and Statistics 3(3), 65-70.
13l
Bain, L'J.
and
Engelhardt, M'
(2006). Introduction
to Probability and Mathematical statistics, 2nd. Duxbury
Press, Belmont, California.
I4l
Bijak, J' (2010). Bayesian Forecasting and Issues
Southampton, I-19.
of
Uncertainty. Centrefor population change, (Jniversity
of
t5l Broemeling,L.and Shaarawy(1988).TimeSer_iesA_BayesianAnalysisinTimeDomain, inBayesian
analysis of Time series and Dynamic Moders, (J.c, spail, ed.; l-2l,MarcelDekker,
New york.
(2004). optimql Statistical decisions.
t6l
DeGroot, M.H'
17l
{an' c' a,nd Yao, s'(2008). Bayesian Approach for ARMA Process and Its Application. International Business
Reseqrch,
John Wiley
&
Son. Inc. New yersey
1(4),49-55.
t8l
Gelman, A.(2008), objections to Bayesian
statistics, Boyesian Anatysis 3(3), 445_ 450.
t9] Geweke, J and whiteman, C., (2004), Bayesian Forecasting,Deparfment
of Economics, University of Iowa.
[10] Hussein, H' M' A' (2008)' Bayesian Analysis_of the Autoregressive-Moving Average Model
with Exogenous Inputs
Using Gibb Sampling. Journar of Appried sciences Reseirch, a(r2),
tss-5-rsg2.
ll
ll
Ismail' M'A' and Amin, A.A. (2010). Gibbs Sampling for SARMA
Models. paper, Faculty of Commerce, statistics
Department, Menoufia (Jniversity, Egpt, l_15.
tl2l
Ismail, M'A' and Charif, H'A. (2003). Bayesian Inference for Thresold
Moving Average Models. METR7NInternqtional Journal ofStatistics, vol.
Ilg_132.
LXI,
tl3l
Kleibergen, F. and Hoek, H.
(1996). Bayesian Analysis of ARMA model using Non informative prior.
I_24
Paper, Econometric Institute, Erasmus University, Rotterdqm,
ll4]
II
Lee,
J'c', Lin, T.I. and Hsu, Y.L' (2004). Bayesian Analysis of Box-cox transformed
linear mixed models with
ARMA(p,q) dependence. Journal of Statiitical Planning and Inference,
t33(2005), qiS-.aSl.
5]
Liu, S. I. ( 1995). Comparison of Forecasts for ARMA Models Between
A Random coefficient Approach and
A Bayesian Approach. Commun. Statist. _ Theory Meth., 24(2),
3l 9_333
tl6l Liu,
S' I. (1995). Bayesian
Multiperiod Forecasts
2 I 1_224.
[17]
Pole, A', west, M' and Harrison , J.
Chapman and Hall, New york.
tl8l
Ramachandran, K'M' and Tsokos, c.P.
Academic press. San Diego, California.
[19]
Safadi, T' and Morettin, P.A.
w (2006).
ARX Models . Ann. Inst. statist. Mqth. vol. 47, no.2,
(lgg4). Apptied Bayesian
(2000)'
Identification
Bayesian
of
Forecasting
and
Time series Analysis.
(2009). Mathemqtical statistics with Applications, Elsevier
Analysis
sankhya: The Indian Journar ofstatistics, vorume
I20l
for
6i,
of
series B,
Threshold Autoregressive Moving Average Models.
pt. 3, 353-37r.
Methods Applied
^u**t9ty'
9''
Intetnational conference on Probqbilistic uethLi-'ippti"d to power to Srreamflow Data.
systems KTH ( l l-15
June 2006, Stockholm, Sweden), t_7
ARMA Model by Bayesian
(t)
o
FcsP
jj
\a
(g
7-E
ER
()rJ
9r,
cg
>8
({< ()
vc)
€>C/]
2!u
==
c.l
\z
^\JL.r
at
cd
TIr
&
C)
.l_)
(-
\/
a
C.)
^7
P
F
63
.Fl
H
tr
,k
C€
C)
c+
f-l
U
&
a1
ts
o
flfltl-.I
t)''v4
k
I
$-r
€tn
=F
;cg9
h€cl
qo".i
NE-^c..
-sxg:*
\'vrx5
lL-
s\
ru
e
'F!
\h
*)trtr
-'1
{t!20.8
Ql
q.s2o
-SLo
*t 'i
-!=:''e'==
-\
r-.,,
i6 E
{_)
6
i-
o
-l
F
bO
=,
rh
--=
-i-t €
SFTfi ?;e
FN
q
Itt
l\
C)
ca
a
rfrrt
E-
r
=
CA
@
s
t{
tv
-)
()
.E
E
e
H
C)
io
r,'l
;
5
E
o
@
r-l
:
r,,:;ffi
;E
F
i)
ll
E
ud
'F{
.\---'-.,
C)
tn
t-i
IJ
'..--c=
o
lt
\\
Lampiran 8
LEMBAR
HASIL PENITAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015)
b. ISBN
c. Tahun Terbit
d. Penerbit
e. Jumlah halaman
f.
Kategori Publikasi Makalah
(beri /pada kategori yang tepat)
Hasil Penilaia n Peer Review
August 2015
USU Medan
WEB Laman
{
l-l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Komponen
Yang Dinilai
Nilai Maksimal Prosiding
lnternasional
Nasaional
{
Nilai Akhir Yang
E
Diperoleh
0,f
d.
Kelengkapan unsur isi artikel (10%)
&f
b.
Ruang lingkup dan kedalaman pembahasan (30%)
2,7
2,f
c.
Kecukupan dan kemutahiran data/informasi dan
metodoloei (30%)
Kelengkapan unsur dan kualitas penerbit (30%)
7,V
2
2,7
2,f
d.
Totat =
f
(lOO%l
/,d
Nilai Pengusul
Medan,
September 2016
Reviewer 1,
\'\
ri
JJ
M.S
986011001
rsitas Sumatera Utara
Prof. Dr. Tulus, M.Si
NrP. 19620901 198803 1002
Unit Kerja : Guru Besar FMIPA USU
Lampiran 8
TEMBAR
HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015)
b. ISBN
c. Tahun Terbit
d. Penerbit
e. Jumlah halaman
f.
Kategori Publikasi Makalah
{beri /pada kategori yang tepat)
Hasil Penilaia n Peer Review
WEB Laman
{
|_l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Komponen
Yang Dinilai
a.
Kelengkapan unsur isi artikel (10%)
b.
Ruang lingkup dan kedalaman pembahasan (30%)
c.
Kecukupan dan kemutahiran data/informasi dan
metodoloei(30%)
d.
Kelengkapan unsur dan kualitas penerbit (30%)
Total =
August 201.5
USU Medan
(LOO%I
Nilai Maksimal Prosiding
lnternasional
Nasaional
./
tr
Nilai Akhir Yang
Diperoleh
0,9
a(
L+
2,t
AT
248
v{
2,+
I
8
Nilai Pengusul
Medan,
September 2016
Reviewer 2,
Mengetahui:
,E€kanF,KlP
N Medan,
Dr. Firmansyah, M.Si
NtP, 19551-0251985031002
Unit Kerjai-Univ. Muslim Nusantara
NrP. 19671110 199303 1003
Unit Kerja: Univ. Muslim Nusantara
Lampiran
I
LEIVIBAR
HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PTER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estirnation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015i
b" tsBN
c. Tahun Terbit
d. Fenerbit
e. Jumlah halaman
f"
Kategori Publikasi Makalah
(beri "'pada kategori yang tepat)
Hasil Penilaian Peer Review
August 2015
USU Medan
WEB Laman
./
|_l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Nilai Maks
Komponen
Yang Dinilai
;:lK"l""s@
b.
c.
d.
l"t"r*si"*a
./
Nasaional
n
Nilai Akhir Yang
Diperoleh
\f
otl
Ruang lingkup dan kedalaman pembahasan (30%)
g,T
L,f
Kecukupan dan kemutahiran data/informasi dan
metodoloei (30%)
Kelengkapan unsur dan kualitas penerbit (30%)
a,T
Ltl
L,T
L
Totat =
(LOO%I
L
Nilai Pengusul
Medan, September 2016
Reviewe 3
NtP. 19570804 198503 1002
Unit Kerja : Guru Besar FMIPA UNIMED
under Normal-Gamma Prior
Zul Amryl'* and Adam Baharum2
1
2
Department of Mathematics, State University of Medan, lndonesia
School of Mathematical Sciences, Universiti Sains Malaysia, Pulau Pinang, Malaysia
*Corresponding
Author: zu l.a ryrrv@gmpil,cop
Presented
On The 3'd International Seminar on Operation Research.
To Celebrate The 50th Department of Mathematics University of
Sumatera Utara, Medan, lndonesia.
August 21-23,20L5.
Bayes Estimation for
ARMA Moder Forecasting under
Normal-Gamma Prior
Zul Amryl,- and Adam Baharum2
2schoo,",T:ffi:ll:1iT:*H',il1,::?:i,"Ji;Jil:,$H:fr1;i,Ti#x'ilMa'aysia
-Corresponding
Abstract
Author: zJl.am{r@gmail.com
This paper presents a Bayesian approach to find the Bayes estimator for the point forecast
and forecast variance
of
ARMA model under normal-gamma prior assumption with quadratic loss function in the form mathematical
expression. The
conditional posterior predictive density be obtained based the posterior under normal-gamma
prior and the conditional
predictive density' whereas the marginal conditional posterior predictive density
be obtain-ed based the conditional posterior
predictive density. Fufihermore, neither the point forecast nor the forecast variance
are both derived from the marginal
conditional posterior predictive density.
Keywords ARMA
1.
Model, Bayes Theorem, Normal-Gamma prior
Introduction
Bayes theorem calculates the posterior distribution as proportional to the product
of a prior distribution and the likelihood
function' The prior distribution is a probability model desiribing the knowledge about ihe parameters
before observing the
currently by the available data [1 ]. Main idea of Baye_sian forecaiting is the prrii.tiu"
distribution of the future given thJfast
data follows directly from the joint probabilistic model. The predictlve distiibution
is derived from the sampling predictive
density, weighted by the posterior dishibution
[4]. This pup"i ir done refer to Liu [15] discussed the Bayesian u"nutyiri, ro,
one-step ahead forecast in ARMA model and Liu
[16] also discussed the multiperiod ftrecast for ARX model employed the
Bayesian approach. others the paper related to this research are Amry and Baharum
[2],Fan& yao [7], Kleibergen & Hoek
[13]' and Uturbey [20] also discussed the Bayesian analysis for ARMA model. This'paper focuses to find the mathematical
expression of the Bayes estimator for the point forecast and forecast variance
of ARMI model under normal-gamma prior
assumption with quadratic loss function
2. Materials and Methods
The materials in this paper are some theories in mathematics and statistics such
as the ARMA model, Bayes theorem,
repeated integration, gamma distribution, and the univariate student's t-distribution.
The method is study of literature by
applying the Bayesian analysis under normal-gamma prior assumption.
2.1. ARMA model
The ARMA (1t, q)model [15] defined by:
y, =f.o y,-,
i=t
where {e'}
parameters.
is
sequence
of i i d
* j=tfe,r, , * r,
(2.1)
'
normal random variables with e,-N(0,rt1,
r>0
and unknown,
0,
and
Q
are
2.2.Bayes theorem
Suppose there are two discrete random variables
atd. )r, then the joint probability function can be written
p( x,y) = p( xl y) p,( y) and the marginal probability density function
of
[18] is:
p*(x)= z, p(x,y) 2,p(x|y) p,( y)
(2.2)
Bayes rule for the conditional p( y I x
is
X
:
)
X
p(
IfIis
ylx)= p(x,y) :
P*(r)
p,(y) _ p(xly) p,(y)
p,(x)
Z,p(xly)pr(y)
p(xly)
continuous, the Bayes theorem can be stated as:
,r
r'
(2.3)
r1=- PQ-2)-P'1! tj p(xly) p,r y )dy
(2.4)
by using the proportional notation (oc which can be expressed by:
)
p(ylx) 6 p(x)y)p,(y)
where p( y I x) is posterior distribution, p(x
I
I
(2.s)
is likelihood function and p,( y )is prior distribution.
2.3. Gamma distribution
A positive random quantity
/
is said to have a gamma distribution with paramet er n
>0
density function [17]:
e@=#ia*'
the notation isQ
- c(n,d)
the mean is
o(a)=]
and
d>0 if it has the probability
expf-4al
(2.6)
is var(Q)=ft
and the variance
2.4. Univariate student-t distribution
A random quantity, x, is said to have a student-t distribution on n degrees
of freedom with mode
r > 0if it has the probability density function [17]:
r( n*1),1
p1x
-
y) l)-1,*6- r
r
.,
r')'
rl;)@:
Thenotationisx-t,(p,r),tnemeanis
r(x)=p
-
p and scale parameter
,,,
|'
(2.7)
l
andthevariance
isvar{x)=J7-,ifn>2
3. Results
3.1. Likelihood Function for ARMA model
The k-step-ahead point forecast ofl ,* t , defined by
:
i(k)=E(Y,*ls]:)
(3.1)
where Sj =(y,, y,,..., y,*o_,)
Based the equation (2.
l)
be obtained residuals as:
e, = y,
-tt,y,-,
-f,o,r,_,
(3.2)
€,: e where r: min(0, p+l-q),
byBox&Jenkins[4],thelikelihoodfunctionfory:(Q1,Q2 ,000t,02,..,0)andr
based sj
By conditioning the first p observations and lerling eo:eo-t:...:
/ r s])x
-\
L\v,
I .1,-r ,f p
,
t,ll
r , p^rl-iLLlr,-Lo,t
-2t,,
one may approximate
t,
,n'k-t'
(3.3)
))j
The equation (3.3) can be expressed as:
/
\
r(p.rlsjJ
where B,
: (y,, !tt,
(n.k-ttp
(
cr , *fl-tllti
z
_1._o_,
I
..., lt+t_p, e6 er_t, ..., €*t_q)
I t=p+r
-
2y'
nrk-t
n+k-
|
ll
y @, a,_,)' ll
L /,8,_,+ t=p.r
t=p. r
J)
(3.4)
By
letting
Yp
!p+t
!n*-2
lp-t
Yp
!n+k,3
/ t
/z
/n+k-t-p
€o
€p*t
€n+k-2
p-t
€,
€n+k-j
U=
€
.'li';'i:l
W:U(.f,and V:Uxo
en+k-t-p
p*t-q € p*z-q
p _c_r
€
where 6,=y,-2d,y,,-ZF,aFj,t:p+l,p+2,...,n, 6,
a,, a,-,,..., a,-oo.
& d, aremaximumlikelihoodestimator of Q, & 0,,and
lutua.a iru'
A,
=y,
-Fr
B,_,
(3.5)
the likelihood function in equation (3.4) can be expressed as:
t\v.r
(n-k-tt-p
{
ll
|
lj
-f*r-r
rrpl-=llyl
2I,A;-zy'v+v'wvll
s))ne,
3.2. Posterior distribution under normal-gamma
(3.6)
prior
According to Broemeling and Shaarawy's suggestion [5], the normal-gamma prior of parameters Pand ris:
€(Y,r)=6,(Y )r).
n
(,(r)
r'f-'*o{-1lr'g*-v'ep-p'ev+pre1.t+2Bj}
'l 2'
,.tl
)
where f, - ,Ut,Gd-')
,
I , - c.tu (a, p) , Q is apositive definite matrix of the order (p + q),
By applying Bayes theorem to equation (3.6) and (3.7), the posterior
!o and r-, is:
of
n+k-t-p'2d+p_t
q.
p:w+e
and. K
=')t*,,ri
+ 1tr e1t +
2
p
andB are parameters.
1= r(v,r1s)), a(v,,)
n(Y,r-t1si
where
a
(
,t n
*pI-jlv,ev-v,
(v+ep)-(y+q1,fv+x)j
r,
(3.8)
.
3.3. Conditional posterior predictive density
Based
on
", = ,, -f,O,r,-,
-fr,",-,
with
e,
- u(o,r-,
)
be obtained f (et1s:,y,r-t )= (zo,
,)r *o{_LrG),}
Ifexpressed in y, is:
LI
f(y,lS),V,r-' 1=(2r, ' ''-t
)' uP\-jl,
P
-LQ,Y,-, -
Based the equation (3.9), be obtained the conditional predictive density
f
(
y.,o
I
s),y,
t -' ) =
(z
o,-' I i
*r{-
P,tr,,f' I
of y , *
(3.e)
p..
}lr. " - fit, r.. 0,, - f ,, n,-,)' }
p
o
l'l
*,'L*, f-f;Lt,.r _l,., t,-o-, _\t,,,*_,
)]
I_
* ': *r{-+l ,.-r -lf.r*.---, .ir,"..--,ll'l
"L
[
Bychanging f4,y,,o-,+f,o,r,*-, to:
\,=i
i-t
)))
(3.1 0)
!n+k-l
!
Qt!*t-t *02!n,*-z t .,.1-dp!,*-p
where
B*k-t=0*u,,
+ete,+k-t t0z€n+*-z +,..+eqe,+k_q
!n+k-2,..., !o+r-p,e,11-1,
-f
en+k-2,
6,.0 | s;, v,
-,
"
: (Ot Q, ,
0,
e
j e2
n+k-2
!n+*-p
0r)
€ n+k-t
€
n+k-z
€
n*-q
: V, B,*O-'
..., e,*o_o), theequation (3.10) can be written as:
*,i
*o
*
*p7-ilf,-r
)
,1
{-
!r[ r,. r -
r,
u,
* _,]']
- zv, a,-* t rn+t
* @, u*r-,)'lj
f ,r *v, Ry 2v7 t,,r_,
* ,iI *pl-ily',-o
,,-r lI
(3.1l)
where R = Bn+r_t@ BI*o_, and (v, n,*o_,)t =v, nv
Basedontheequation(3.8)and(3'll),beobtainedtheconditionalposteriorpredictivedensity
f o(t,*ls:,,Y,r-t
)
n o(v,
'"
ofy,t
lsj ) f (y,.ols),v)1 I
B,u-, y*o)- f)
,-o-,-o-r".0-, I lv'av -Yr 1lt
T'
*ofil(y +ep)r BI**., !,.* 1v * yl.r * rcll
+Q1t+
0.,
t"Lll
whereG:P+R
+
(3.12)
3.4. Marginal conditional posterior predictive density
The marginal conditional posterior predictive density
predictive density in equation (3.12):
_f
,(t,.rls,)
=
I[
_:-
=
-f ,f
of Y ,* r be obtained by integrating
the conditional posterior
t,,rls:,y,r1) dydr
ii''-*'-{-
;l:;,o.*r;'.' :;r':)l ;; i;-,:r' : -)}
n
*
o,
z--' ^'_l(r_o-,(1tt+g1t1+n,+k.trn+k)), c(v_c_,(1v+e1t)+8,*r,y,_r)\_f ,...,
* J71,--x-4:]:!_,
J'
o'
"-'l'i, )nu,* B,t*-t !n-t ), c-, (rt +gpt* B,-*-r
t,,* )*y,,,0 + K
)o'
(n+k-l- p+2d )+l
2
*l'.0-'-0.'"
*u -Q - Bl,o-,
G-t
B,*r-t
_K-{V+ep)t
+
(n+ k
- t - p+ 2")
U
f'
(t:,0-, c-, g, *
p1.,1)]
(3.1 3)
Gotv+gp1
-
B|*, B,;J
The marginal conditional posterior predictive density of y,*1
is a univariate student,s t_distribution on (n+k_l_p+2a)
degrees of freedom with mode
(t
a1-o-,
G-' Bn*o-,)" (u:.r-, c' 1v + g1t1) and scale parameter
tr= -
_
T=
(n + k
K-(V+eu)rG^(V+Ou)
v\
-"
- t- p+ za)Q -
ffi+;J'
where
co =G-t *Q
-
Bl.o-,G-'8,-r-,)-'
(c-'ao
'1
3.5. Point forecast and forecast variance
For quadratic loss function, the point forecast of Y
, *p is the posterior mean of the marginal conditional posterior predictive,
that is:
a(a., t s; )=(, -
Bf,*o_, G-,
B,*o_,)-' (r:.0_, c-' 1v + gp 1)
(3.14)
and the forecast variance
of
I,
* r is:
K-(V+
(n+k-1- p+2a)
r*(v,,.r 1s')=
1r
Golv +gpy
+k-l-p+2a
=(n + k
- 3-
p
+ za)-' (x - p,
+
gp 1, c o1v
+
gp 1)
Q
- ul,*,
G -,
B,*o_,),
(3. 1 5)
4. Conclusion and computational procedure
This paper analyzes how to find mathematical expression ofthe point forecast in equation (3.14)
and the forecast variance
in equation (3.15) based the marginal conditional posterior prediitive density in aquation
The conditional posterior
f:.r:1.
predictive density be obtained by multiplying the normal-gamma prior to the condiiional piedictive
density. By iniegrating
the conditional posterior predictive density to paramaters Y and t respectively, obtained
tire marginal .onditionut po-sterior
predictive density. Furthermore, the point forecast and the forecast uuiiun.. are derived
based the mean and the variance of
marginal conditional posterior predictive density that has the univariate student's t-distribution. procedure
to compute the
point forecast and the forecast variance are as follows:
Defines:
Yp
!p+t
!n+r-:
lpt
Yp
!n+k,3
!t
y.
,vh+x t_p
::::
U=
ao
6
€
o_,
iii.
iv.
XO:
€n+k-2
o
€ n+k-3
p*z-q
en+k-t-q
:::
e
ll.
p*l
€
€
l';:.:,1
l:l
lr..r
I
It, Q, a and
B
B,* -, =\y *o -,' !.nr-l,..., / n+t-t-p, € n+k,t,
€ n+*-2,
..., r,*-,u)
Compute:
v.
W:ULf
vi.
V:Ux6
P:IY+Q
^.f'
viii. K =
yl * p'ep + 2p
vii.
ix.
R= B,,r_,& BI*_,
x.
G:P+R
xi.
G-1
xii.
Go
xiii.
xiv.
= G-t + Q - Bl.o_,G-'a,,r_,)-'(c-'nc-')
E(y,.k s: )
)
:
Q
- a:.r-, G-' B,*0,,)-' (u:.,,, c-, 1v + gp 1)
va,(r.ul $) = (, + k - 3 - p + 2a)-'
(*
- f n + ep
f
G,(v + ep ))
Q
-
B!,,r_, G-,
B.*_,)-,
References
lll
Alba, E. And Mendoza, M. Bayesian Forecasting Methods for Short
Time Series. Article
l2l
Amry, Z' and Baharum, A. (2015)' Bayesian Multiperiod Forecasting
for Arma Model under Jeftey,s prior.
Mathematics and Statistics 3(3), 65-70.
13l
Bain, L'J.
and
Engelhardt, M'
(2006). Introduction
to Probability and Mathematical statistics, 2nd. Duxbury
Press, Belmont, California.
I4l
Bijak, J' (2010). Bayesian Forecasting and Issues
Southampton, I-19.
of
Uncertainty. Centrefor population change, (Jniversity
of
t5l Broemeling,L.and Shaarawy(1988).TimeSer_iesA_BayesianAnalysisinTimeDomain, inBayesian
analysis of Time series and Dynamic Moders, (J.c, spail, ed.; l-2l,MarcelDekker,
New york.
(2004). optimql Statistical decisions.
t6l
DeGroot, M.H'
17l
{an' c' a,nd Yao, s'(2008). Bayesian Approach for ARMA Process and Its Application. International Business
Reseqrch,
John Wiley
&
Son. Inc. New yersey
1(4),49-55.
t8l
Gelman, A.(2008), objections to Bayesian
statistics, Boyesian Anatysis 3(3), 445_ 450.
t9] Geweke, J and whiteman, C., (2004), Bayesian Forecasting,Deparfment
of Economics, University of Iowa.
[10] Hussein, H' M' A' (2008)' Bayesian Analysis_of the Autoregressive-Moving Average Model
with Exogenous Inputs
Using Gibb Sampling. Journar of Appried sciences Reseirch, a(r2),
tss-5-rsg2.
ll
ll
Ismail' M'A' and Amin, A.A. (2010). Gibbs Sampling for SARMA
Models. paper, Faculty of Commerce, statistics
Department, Menoufia (Jniversity, Egpt, l_15.
tl2l
Ismail, M'A' and Charif, H'A. (2003). Bayesian Inference for Thresold
Moving Average Models. METR7NInternqtional Journal ofStatistics, vol.
Ilg_132.
LXI,
tl3l
Kleibergen, F. and Hoek, H.
(1996). Bayesian Analysis of ARMA model using Non informative prior.
I_24
Paper, Econometric Institute, Erasmus University, Rotterdqm,
ll4]
II
Lee,
J'c', Lin, T.I. and Hsu, Y.L' (2004). Bayesian Analysis of Box-cox transformed
linear mixed models with
ARMA(p,q) dependence. Journal of Statiitical Planning and Inference,
t33(2005), qiS-.aSl.
5]
Liu, S. I. ( 1995). Comparison of Forecasts for ARMA Models Between
A Random coefficient Approach and
A Bayesian Approach. Commun. Statist. _ Theory Meth., 24(2),
3l 9_333
tl6l Liu,
S' I. (1995). Bayesian
Multiperiod Forecasts
2 I 1_224.
[17]
Pole, A', west, M' and Harrison , J.
Chapman and Hall, New york.
tl8l
Ramachandran, K'M' and Tsokos, c.P.
Academic press. San Diego, California.
[19]
Safadi, T' and Morettin, P.A.
w (2006).
ARX Models . Ann. Inst. statist. Mqth. vol. 47, no.2,
(lgg4). Apptied Bayesian
(2000)'
Identification
Bayesian
of
Forecasting
and
Time series Analysis.
(2009). Mathemqtical statistics with Applications, Elsevier
Analysis
sankhya: The Indian Journar ofstatistics, vorume
I20l
for
6i,
of
series B,
Threshold Autoregressive Moving Average Models.
pt. 3, 353-37r.
Methods Applied
^u**t9ty'
9''
Intetnational conference on Probqbilistic uethLi-'ippti"d to power to Srreamflow Data.
systems KTH ( l l-15
June 2006, Stockholm, Sweden), t_7
ARMA Model by Bayesian
(t)
o
FcsP
jj
\a
(g
7-E
ER
()rJ
9r,
cg
>8
({< ()
vc)
€>C/]
2!u
==
c.l
\z
^\JL.r
at
cd
TIr
&
C)
.l_)
(-
\/
a
C.)
^7
P
F
63
.Fl
H
tr
,k
C€
C)
c+
f-l
U
&
a1
ts
o
flfltl-.I
t)''v4
k
I
$-r
€tn
=F
;cg9
h€cl
qo".i
NE-^c..
-sxg:*
\'vrx5
lL-
s\
ru
e
'F!
\h
*)trtr
-'1
{t!20.8
Ql
q.s2o
-SLo
*t 'i
-!=:''e'==
-\
r-.,,
i6 E
{_)
6
i-
o
-l
F
bO
=,
rh
--=
-i-t €
SFTfi ?;e
FN
q
Itt
l\
C)
ca
a
rfrrt
E-
r
=
CA
@
s
t{
tv
-)
()
.E
E
e
H
C)
io
r,'l
;
5
E
o
@
r-l
:
r,,:;ffi
;E
F
i)
ll
E
ud
'F{
.\---'-.,
C)
tn
t-i
IJ
'..--c=
o
lt
\\
Lampiran 8
LEMBAR
HASIL PENITAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015)
b. ISBN
c. Tahun Terbit
d. Penerbit
e. Jumlah halaman
f.
Kategori Publikasi Makalah
(beri /pada kategori yang tepat)
Hasil Penilaia n Peer Review
August 2015
USU Medan
WEB Laman
{
l-l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Komponen
Yang Dinilai
Nilai Maksimal Prosiding
lnternasional
Nasaional
{
Nilai Akhir Yang
E
Diperoleh
0,f
d.
Kelengkapan unsur isi artikel (10%)
&f
b.
Ruang lingkup dan kedalaman pembahasan (30%)
2,7
2,f
c.
Kecukupan dan kemutahiran data/informasi dan
metodoloei (30%)
Kelengkapan unsur dan kualitas penerbit (30%)
7,V
2
2,7
2,f
d.
Totat =
f
(lOO%l
/,d
Nilai Pengusul
Medan,
September 2016
Reviewer 1,
\'\
ri
JJ
M.S
986011001
rsitas Sumatera Utara
Prof. Dr. Tulus, M.Si
NrP. 19620901 198803 1002
Unit Kerja : Guru Besar FMIPA USU
Lampiran 8
TEMBAR
HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PEER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estimation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015)
b. ISBN
c. Tahun Terbit
d. Penerbit
e. Jumlah halaman
f.
Kategori Publikasi Makalah
{beri /pada kategori yang tepat)
Hasil Penilaia n Peer Review
WEB Laman
{
|_l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Komponen
Yang Dinilai
a.
Kelengkapan unsur isi artikel (10%)
b.
Ruang lingkup dan kedalaman pembahasan (30%)
c.
Kecukupan dan kemutahiran data/informasi dan
metodoloei(30%)
d.
Kelengkapan unsur dan kualitas penerbit (30%)
Total =
August 201.5
USU Medan
(LOO%I
Nilai Maksimal Prosiding
lnternasional
Nasaional
./
tr
Nilai Akhir Yang
Diperoleh
0,9
a(
L+
2,t
AT
248
v{
2,+
I
8
Nilai Pengusul
Medan,
September 2016
Reviewer 2,
Mengetahui:
,E€kanF,KlP
N Medan,
Dr. Firmansyah, M.Si
NtP, 19551-0251985031002
Unit Kerjai-Univ. Muslim Nusantara
NrP. 19671110 199303 1003
Unit Kerja: Univ. Muslim Nusantara
Lampiran
I
LEIVIBAR
HASIL PENILAIAN SEJAWAT SEBIDANG ATAU PTER REVIEW
KARYA ILMIAH : PROSIDING INTERNASIONAL
"Bayes Estirnation for ARMA Model Forecasting Under Normal-Gamma
Judul Makalah
Prior"
Penulis Makalah
Zul Amry and, Adam Baharum
ldentitas Makalah
a. Judul Prosiding
lnternational Seminar on Operational Research
(lnteriOR 2015i
b" tsBN
c. Tahun Terbit
d. Fenerbit
e. Jumlah halaman
f"
Kategori Publikasi Makalah
(beri "'pada kategori yang tepat)
Hasil Penilaian Peer Review
August 2015
USU Medan
WEB Laman
./
|_l
Prosiding Forum llmiah lnternasional
Prosiding Forum llmiah Nasional
:
Nilai Maks
Komponen
Yang Dinilai
;:lK"l""s@
b.
c.
d.
l"t"r*si"*a
./
Nasaional
n
Nilai Akhir Yang
Diperoleh
\f
otl
Ruang lingkup dan kedalaman pembahasan (30%)
g,T
L,f
Kecukupan dan kemutahiran data/informasi dan
metodoloei (30%)
Kelengkapan unsur dan kualitas penerbit (30%)
a,T
Ltl
L,T
L
Totat =
(LOO%I
L
Nilai Pengusul
Medan, September 2016
Reviewe 3
NtP. 19570804 198503 1002
Unit Kerja : Guru Besar FMIPA UNIMED