Masoud 2014,... The performance evaluation of the model can be described below:
3.1 Statistical Performance Evaluation of the Model
In order to estimate the forecasting statistical performance of some methods or to compare several methods we should define error functions. Many
previous research works had applied some of the following forecast accuracy measures: Mean Error ME, Mean Absolute Error MAE, Mean
Squared Error MSE, Root Mean Squared Error RMSE, Standard Deviation of Errors SDE, Mean Percent Error MPE and Mean Absolute
Per cent Error MAPE, etc. In our study we use four performance criteria namely mean absolute error MAE, root mean square error RMSE, mean
absolute percentage error MAPE and goodness of fit R
2
. The back- propagation learning algorithm was used to train the three-layered feed-
forward ANN structure in this study were the most used error functions is as following:
The mean absolute error is an average of the absolute errors E = P
i
- , where P
i
and are the actual or observed value and predicted value, respectively. Lesser values of these measures show more correctly predicted
outputs. This follows a long-standing tradition of using the “ex-post facto” perspective in examining forecast error, where the error of a forecast is
evaluated relative to what was subsequently observed, typically a census based benchmark Poon 2005. The most commonly used scale-dependent
3.1 Statistical Performa ma
n nce Evaluation of the
Mo M
del
In order er
t to estimate the for
or ec
c as
s ti
t ng
ng statist
t ic
i al performan
ce ce
of some methods or
or to compare
e se
e ve
ve ra
r l meth
h ods we sho
ho ul
ul d
d de
defi fi
n ne error fun
nct c
ions. Many previo
io us
us r r
e esearch
h w
work s
ha d
ap plied
so o
me me
of th he
e fo fo
ll ll
owing forecast
ac accu
cura ra
cy m m
ea sure
s: Mean Erro
r M E, Mean Abs
ol ut
ute e
Erro ro
r M
MAE AE
, M Mean
Sq S
uare re
d Error MSE, Root
Me an Squar
ed Error
R RMSE
E ,
, S Standar
rd De
e v
vi ation of Errors S
DE , Mea
n Percen
t Er
ror MPE and M
Mean n
A A
bs bs
ol o
ute P
Per cent
E rr
or MAPE
, etc
. In
o ur
s tu
dy we us
e fo ur
perfo rm
m ance
c criteri
ria a
n na
mely mean absolu te err
or MAE,
root mea
n square error RM
MSE, m mean
n a
ab so
lute perce nt
ag ag
e e
er er
ro r
r MAPE and g g
oo oo
dn dn
ess of fit R
2
. T
T h
he bac ac
k- k-
propagation learning algorit ithm
hm w wa
as used to train the three-layered f
fee ee
d d-
forward ANN structure in this study were the most used error fun n
ct ct
i ion
ns i is
s as
fo fo
ll ll
ow ow
in in
g: g:
Th Th
e me
mean absolute error is a a
n averag age of the absolute er
er ro
ro rs
rs E
E = P
i
- , where P
i
and are the
e actual or observed value and predicted value,
d respectively. Lesser values o
of these m measures show more correctly predicted
outputs. This follows a long-st tand
ding tradition of using the “ex-post facto”
summary measures of forecast accuracy are based on the distributions of absolute errors |E| or squared errors E
2
observations n is the sample volume. The mean absolute error is given by:
Mean Absolute Error MAE =
¸ ¹
· ¨
© §
¦
n i
n E
1
i = 1, 2,…n 6
The MAE is often abbreviated as the MAD “D” for “deviation”. Both MSE and RMSE are integral components in statistical models e.g., regression.
As such, they are natural measures to use in many forecast error evaluations that use regression-based and statistical. The square root of the mean
squared error as follows:
Mean Square Error MSE =
¸ ¹
· ¨
© §
¦
n i
n E
1 2
i = 1, 2,…n
Root Mean Square Error RMSE =
¸ ¹
· ¨
© §
¦
n i
n E
Sqrt
1 2
i = 1, 2,…n 7 If the above RMSE is very less significant, the prediction accuracy of the
ANN model is very close to 100. Since percentage errors are not scale- independent, they are used to compare forecast performance across different
data sets of the area using absolute percentage error given by APE = P
i
- 100. Like the scale dependent measures, a positive value of APE is
derived by taking its absolute value | APE | observations n. This measure includes:
MAPE =
¸ ¹
· ¨
© §
¦
n i
n APE
1
i = 1, 2,…n 8
volume. The mean absol ol
ut ute error is g
iv iv
en en
by:
Mean A A
bs olute Error M
M AE =
¸ ¹
¸¸·¸¸ ¨
© ©
¨¨§¨¨
¦
n i
i
n
1 1
i i = 1, 2,…n
6 T
The MAE is of fte
ten n
ab ab
breviated as the E
MAD “ “
D” D”
f f
or “de de
viation” ”
. .
Both MSE and RM
RM SE
SE are i
i nt
nt egral compon
en ts in statis
ti i
ca cal
l mode
d ls
ls e.
e g.
g , regre
ession. E
As As
s such,
, t
t h
he y are
na tural measur
es to use in m
an y fo
re ca
ca st e
rr rr
or or e
ev valuat
io io
ns that
u use
regressio n-
based and st
atistical. T
he square
ro o
ot o
o o
f f
th the
e mean
n sq
q u
ua red
er ror as follows
:
Mean Square Erro r
M SE
=
¸ ¹
¸¸ ·
¸¸ ¨
© ¨¨
§¨¨
¦
n i
n
1
i = 1, 2, …
…n
Root Mean Square Error r
RM RMSE
E =
=
¸ ¹
¸¸ ·
· ¸
¸ ¸
¸ ©
§ §
¨ ¨
¨ ¨
¦ ¦
n i
n E
S Sqrt
¨ ©
¨¨§¨¨
¦
1
i = 1, 2,…n 7
7 If
If t
t he
h above RMSE is very less significant,
, the pr
p ediction a
a cc
cc u
urac c
y y of
of t
the ANN model
l is
is v v
er y
y close to
to 1
100 00.
. Since p
p erce
cent ntag
ag e errors are
e n
not ot
s cale-
in in
de de
pe pe
nd nd
en en
t t, they are used t
t o
o compar are forecast perform
m an
an ce
ce a
a cr
cr os
s different data sets of the area using a
absolute per rcentage error given by APE = P
i
- 100. Like the scale dep
pendent m
measures, a positive value of APE is derived by taking its absolute v
value ue | APE | observations n. This measure
E i
l d
i = 1,2,...n 10
The use of absolute values or squared values prevents negative and positive errors from offsetting each other. All these features and more make
MATLAB an indispensable tool for use in this work.
Goodness of Fit R
2
=
¸ ¹
· ¨
© §
¦
n i
e E
1 2
2
i= 1, 2,…n 9
where e
i
= p
i
- p
i
, is the forecast error values. p
i
, the actual values and p
i
, denote the predicted values. The more R
2
correlation coefficient gets closer to one, the more the two data sets are correlated perfectly. As the aim
of all of the prediction system models proposed in this study is to predict the direction of the stock price index forecasting, the correlation between the
outputs do not directly reflect the overall performance of the network.
3.2 Financial Performance Evaluation of the Model