Testing Normality and Bandwith Estimation Using Kernel Method For Small Sample Size
KERNEL METHOD
K nh h x f
= ∑
− − ≠
u u n n b b i i h u w
1 ( 1)
1
( ) *
(2.2)
, for all i ≠ i* = 1, 2, ..., n. The kernel estimates of h 1 , h 2 and h respectively are:
u ii* and v ii*
,X 2 ,…,X n be a random sample of density function F with density f. and let u ii* = X i + X i* with density h 1 and v ii* = X i + X i* with density h 2 furthermore, let h(u, v) as joint density of
X 1
(2.1) for Xi= xi, i=1,2…,n. The kernel function K represents how the probaility mass assigned, so for histogram it is just a constant interval, wich satistisfied . The smoothing funtion hn is Let
1 ^ 1 ) ; (
⎝
⎛ −
= n i i h X x62 Testing ......... (Netti Herawati & Khoirin Nisa)
⎟ ⎠ ⎞ ⎜
∑ =
Let X 1 ,X 2 ,…,X n be a sample, we write the density estimator
or bandwiths which analogous to the bin width in a histogram. The problem of choosing the choose the estimate that is most in accordance with one’s prior ideas about the density Jones (1984), bandwith is scale factor to control the spread out of point observation in the curve. In this article, we tested normality assumption using kernel method and estimated the optimal bandwith using Kullback-Leibler cross validation method for small sample sizes. In order to get the satisfying result we also use Kolmogorof-Smirnov goodness of fit test to compare with result obtained from kernel method.
n
) assumption on the errors is satisfied, this plot should look like a sample from normal distribution centered at zero. Unfortunately with small samples, considerable fluctuation in shape of histogram often occurs, so the appearance of moderate departure from normality does not necessarily imply a serious violation of assumption. However, gross deviations from normality are potentially serious and required further analysis Another way to test normality assumption in parametric method can be done by using normal probability plot of residulas. In nonparametric method there are also some procedures to test normality such as Shapiro- Wilk tests, Locke-Spurrier test, etc. However such tests require special tables. Kernel method is considered as nonparameteric method. In kernel method, the idea is based on density estimator by more fairly spreading out the probability mass of each observation, not arbitrarily in a fixed interval, but smoothly around the observation, In order to smooth around the observation, it is important to choose what is called smoothing function h
2
σ
In many statistical inference the statistics test is ussually based on assumption of normal distribution. A check of normality assumption could be made by plotting a histogram of the residual. If the NID(0,
INTRODUCTION
This article aimed to study kernel method for testing normality and to determine the density function based on curve fitting technique (density plot) for small sample sizes. To obtain optimal bandwith we used Kullback- Leibler cross validation method. We compared the result using goodness of fit test by Kolmogorof Smirnov test statistics. The result showed that kernel method gave the same performance as Kolmogorof Smirnov for testing normality but easier and more convinient than Kolmogorof Smirnov does. Keywords: Normality, kernel method, bandwith, Kolmogorof Smirnov test
Department of Mathematics FMIPA University of Lampung ABSTRACT
Testing Normality and Bandwith Estimation Using Kernel Method For Small Sample Size
- ˆ ( ) ii
- ˆ( , ) ii ii
Jurnal ILMU DASAR Vol. 10 No. 1. 2009 : 62 – 67
1 (2.11)
25. The estimating optimal bandwidth using Kullback-Leibler cross validation method was done by using S-Plus software. We compare the result with the Kolmogorov-Smirnov Goodness of Fit test statistics..
To demonstre the method introduced we simulated the data from N(0,1), N(5,10), exponential ditribution, and Gamma distribution with the size of samples are 10 and
α
= max | F n (x) – F(x)| > D
D
n(x) We reject H if
: F(x) ≠ F
H : F(x) = F (x), ( ∀x)
H
1Kolmogorov dan Smirnov (1948) goodness of fit test is used to test:
Kolmogorov-Smirnov
= h max = max CV KL (h) ∑ log
We obtained h oversmoothing bandwidth or
h
KL(2.10) According to Hardle (1991), the optimal bandwidth h is h which maximized (CV KL )
∑ ∑ = ≠
⎝ ⎛ − =
⎟⎟ ⎠ ⎞ ⎜⎜
⎢ ⎢ ⎣ ⎡
1 1 − − ⎥ ⎥ ⎦ ⎤
X X K n n i i j j i ) 1 ( log log
h
[ ] h n
Normal distribution ((N(0,1)) with n = 10
h
osCV h 1 ) ( ˆ log
u u v v b b n n b i i h u v w w
= − ∫ ∫
δ ∞ ∞ −∞ ∞
2 ˆ ˆ ˆ ˆ [ ( , ) ( ) ( )] . h u v h u h v dudv
1
2
∫ ∫
∞ ∞ −∞ ∞ = −
2 1 2 [ ( , ) ( ) ( )] . h u v h u h v dudv δ
= ∑
− − − ≠
1 ( 1)
= 0.53 and CV
( ) ( ) * * 2
= 0.65 and CV KL (h) maximum = 0.7. Using the estimating curve with bandwith (h) 0.45 to 0.7, it showed that the optimal bandwith (h opt )
os
We obtained h oversmoothing bandwidth or h
Normal distribution ((N(0,1)) with n = 25
(h) maximum= 0.6, and the estimated curve is shown in Figure 1.
KL
) is is the same as CV
opt
(h) maximum = 0.6. Using the estimating curve with bandwith (h) 0.3 to 0.6, it showed that the optimal bandwith (h
KL
1 ) (
= n i KL i h X f n
63 and (2.4) with b = b n is a positive constant called bandwith and w(.). is a known symmetric bounded density called kernel. Therefore, we can assume w has mean 0 and a finite variance µ 2 (w), and that b
μ,σ 2
X n
Suppose that an independent observation X 1 , X 2 , …,
Kullback-Leibler cross validation method
random sample X 1 , X 2 , …, X n and using estimates , and above, we can performe the normality test based on the assumption of:
h 1 (u)h 2 (v) u and v are independent. Using the
This is so, since H0 is equivalent to H : h(u, v) =
> 0. A measure of departure from H is: (2.5)
μ Є R and σ 2
) , for
) agints H 1 : f isnot N(
[ ] ∑ =
μ,σ 2
→ 0 as n → ∞. If we want to test normality assumption such as: H : f is N(
∏ ∑ ∏ = ≠ ⎟⎟ ⎠ ⎞ ⎜⎜
⎝ ⎛ − −
= = n i n j i j i n i i i h i h
X X K h n X f
X L 1 , ) 1 (
1 ) ( ˆ ) (
(2.9) times 1/n, we get Kullback-Leibler cross validation
( ) KL CV
:
from f were available. Then the likelihood of f as the density underlying the observation X would be log f(X), regarded as a function of h as the log likelihood of the smoothing parameter h. Likelihood Cross Validation (LCV) is average over each choice of ommited Xi, to give the score:
RESULTS AND DISCUSSION
⎝ ⎛ = ⎟
is the value of h which maximized the Kullback-Leibler information distance and h os is h oversmoothing bandwidth.
(2.8) This estimate is called cross validation defined by:
ˆ ,
) 1 ( 1 ) (
X X K h n X f
− − = n j i j i i i h h
⎟⎟ ⎞ ⎜⎜ ⎛
∑ ≠
, , so that with counting from is the same as getting likelihoof function for Xi (Hardle,1991).
Statistics value for different h will guide us to get better h,, because the algorithn of this statistics approximately close to
For instance, we would like to test independent random variable Xi, Likelihood of Xi is ∏ .
, where h opt
⎟ ⎠ ⎞ ⎜
(2.7) To estimate the optimal bandwidth can be done by minimized h opt and h os
^
∧ , log
( ) i X f n LCV ^ 1
log
∑ −
= (2.6)
From (2.6) can be seen that the value of h maximized LCV(h). The maximum LCV(h) can be obtained from Kullback Leiebler information distance, defined by:
( ) ( ) dx x f x f f f f d
KL h ∫
⎠ ⎞ ⎜ ⎝ ⎛
64 Testing ......... (Netti Herawati & Khoirin Nisa)
We obtained h oversmoothing bandwidth (h
Figure 1. Normal density curve (N(0,1)) with n=10.
(h) maximum = 6, and the estimated curve is shown in Figure 4.
KL
) is the same as CV
opt
(h) maximum = 6. Using the estimating curve with bandwith (h) 3 to 6, it showed that the optimal bandwith (h
KL
) = 5.3 and CV
os
Normal distribution ((N(5,10)) with n = 25
Normal distribution ((N(5,10)) with n = 10
(h) maximum = 5, and the estimated curve is shown in Figure 3.
KL
) is the same as CV
opt
(h) maximum = 5. By using the estimating curve with bandwith (h) 3 to 5, it showed that the optimal bandwith (h
KL
CV
) = 4.4 and
os
We obtained h oversmoothing bandwidth (h
Figure 2. Normal density curve (N(0,1)) with n=25.
Jurnal ILMU DASAR Vol. 10 No. 1. 2009 : 62 – 67
(h) maximum = 2, and the estimated curve is shown in Figure 7.
We obtained h oversmoothing bandwidth (h
os
) = 1.52 and CV
KL
(h) maximum = 0.5. Using the estimating curve with bandwith (h) 1 to 2, it showed that the optimal bandwith (h
opt
) is the same as CV
KL
Gamma distribution (G(1,1)) with n = 25
(h) maximum = 0.5, and the
We obtained h oversmoothing bandwidth (h
os
) = 0.63 and CV
KL
(h) maximum = 0.7. Using the estimating curve with bandwith (h) 0.3 to 0.7, it showed that the optimal bandwith (h
opt
) is the same as CV
KL
Gamma distribution (G(1,1)) with n = 10
KL
65 Figure 3. Normal density curve (N(5,10)) with n=10.
) is the same as CV
Figure 4. Normal density curve (N(5,10)) with n=25.
Exponential distribution ((E(1)) with n = 10
We obtained h oversmoothing bandwidth (h
os
) = 0,97 and CV
KL
(h) maximum = 1. Using the estimating curve with bandwith (h) 0.5 to 1, it showed that the optimal bandwith (h
opt
KL
) is the same as CV
(h) maximum = 1, and the estimated curve is shown in Figure 5.
Exponential distribution ((E(1)) with n = 25
We obtained h oversmoothing bandwidth (h
os
) = 0.49 and CV
KL
(h) maximum = 0.5. Using the estimating curve with bandwith (h) 0.2 to 0.5, it showed that the optimal bandwith (h
opt
(h) maximum = 0.7, and the estimated curve is shown in Figure 8.
66 Testing ......... (Netti Herawati & Khoirin Nisa) Figure 5. Exponential density curve (E(1)) with n=10.
Figure 6. Exponential density curve (E(1)) with n = 25.
Jurnal ILMU DASAR Vol. 10 No. 1. 2009 : 62 – 67
67 Table 1. Kolmogorov-Smirnov Goodness of Fit Test Distribution
N(0,1) N(5,10) Exponential (1) Gamma (1,1) Sample D D D D D D D D
n 0.05 n 0.05 n 0.05 n
0.05
size
ns ns * *
n=10 0.1443 0.369 0.1081 0.369 0.3920 0.369 0.4438 0.369
- ns
- ns
n=25 0.0879 0.283 0.0776 0.283 0.2843 0.283 0.2772 0.283 Note: ns= nonsignificant at
α=0.05
- =significant at
α=0.05
Kolmogorov-Smirnov goodness of Fit test departure from normality can be detected easily
The result of Kolmogorov-Smirnov Goodness using kernel method. By comparing the results of Fit Test can be seen in Table 1. We reject from Kernel Method and Kolmogorov Smirnov hipotesis nul when D > D . test, we conclude that the two test gave the
n
0.05 By comparing the figures obtained by same performance
Kernel Method with Kolmogorof Smirnov test, we can say that both methods gave the same
REFERENCES
preformance. Figure 1, Figure 2, Figure 3, and Figure 4 showed that the data were normally
distributed which were the same as
Kolmogorof Smirnov’s (Table 1). The same result were given by Figure 5 and Figure 6
(data from exponential distribution) as well as
Figure 7 and Figure 8 (data from Gamma distribution) when comparing with Kolmogorof Smirnov’s (Table 1).
The result is also the same as the result by
the result of Locke & Spurrier (1976) when
simulated from distribution different than
normal such as from the Chi-Square, the Cauchy and the Beta Distributions.
CONCLUSION
The simulation study illustrates that kernel method is useful for testing normality for n=10
and n=25. This study also reveals that severe