Directory UMM :Journals:Journal_of_mathematics:GMJ:
Georgian Mathematical Journal
1(1994), No. 2, 197-212
LIMIT DISTRIBUTION OF THE INTEGRATED SQUARED
ERROR OF TRIGONOMETRIC SERIES REGRESSION
ESTIMATOR
E. NADARAYA
Abstract. Limit distribution is studied for the integrated squared
error of the projection regression estimator (2) constructed on the
basis of independent observations (1). By means of the obtained
limit theorems, a test is given for verifying the hypothesis on the
regression, and the power of this test is calculated in the case of
Pitman alternatives.
Let observations Y1 , Y2 , . . . , Yn be represented as
Yi = µ(xi ) + εi ,
i = 1, n,
(1)
where µ(x), x ∈ [−π, π], is the unknown regression function to be estimated
by observations Yi ; xi , i = 1, n, are the known numbers, and −π = x0 <
x1 < · · · < xn ≤ π, εi , i = 1, n, are independent equidistributed random
variables; Eε1 = 0, Eε21 = σ 2 , and Eε41 < ∞.
The problem of nonparametric estimation of the regression function µ(x)
for the model (1) has a recent history and has been treated only in few
papers. In particular, a kernel estimator of the Rosenblatt–Parzen type for
µ(x) was proposed for the first time in [1].
Assume that µ(x) is representable as a converging series in L2 (−π, π)
with respect to the orthonormal trigonometric system
o∞
n
.
(2π)−1/2 , π −1/2 cos ix, π −1/2 sin ix
i=1
Consider the estimator of the function µ(x) constructed by the projection
method of N.N. Chentsov [2]
N
a0n X
µnN (x) =
ain cos ix + bin sin ix,
+
2
i=1
1991 Mathematics Subject Classification. 62G07.
197
(2)
198
E. NADARAYA
where N = N (n) → ∞ for n → ∞ and
ain =
n
n
1X
1X
Yj ∆j cos ixj , bin =
Yj ∆j sin ixj ,
π j=1
π j=1
∆j = xj − xj−1 , j = 1, n,
i = 0, N .
The estimator (2) can be rewritten in a more compact way as
µnN =
n
X
j=1
where KN (u) =
1
2π
P
Yj ∆j KN (x − xj ),
eiru is the Dirichlet kernel.
|r|≤N
In [3], p.347, N.V. Smirnov considered estimators of the type (2) for
a specially chosen class of functions µ(x) in the case of equidistant points
xj ∈ [−π, π] and of independent and normally distributed observation errors
εi . In [4] an estimator of the type (2) is obtained, which is asymptotically
equivalent to projection estimators which are optimal in the sense of some
accuracy criterion. The asymptotics of the mean value of the integrated
squared error of the estimator (2) is considered in [5].
It is of interest to investigate the limit distribution of the integrated
squared error
Z
π
−π
[µnN (x) − µ(x)]2 dx,
which is the goal pursued in this paper. The method used to prove the
theorems below is based on the functional limit theorem for a sequence of
semimartingales [6].
Denote
Z π
2
n
µnN (x) − EµnN (x) dx,
UnN =
2π(2N + 1) −π
2
Qir = ∆i ∆r KN (xi − xr ), σnN
=
ηik =
r−1
n X
X
n2 σ 4
Q2 ,
π 2 (2N + 1)2 r=2 j=1 jr
n
εi εk Qik ,
π(2N + 1)σnN
ξ1 = 0, ξk =
k−1
X
ηik ,
k = 2, n, ξk = 0, k > n,
i=1
and assume that Fk is σ-algebra generated by random variables ε1 , ε2 ,
. . . , εk , F0 = (φ, Ω).
Lemma 1 ([7], p.179). The stochastic sequence (ξk , Fk )k≥1 is a martingale-difference.
LIMIT DISTRIBUTION OF THE SQUARED ERROR
199
Lemma 2. Let p(x) be the known positive continuously differentiable
density on [−π, π], and points xi be chosen from the relation
Rdistribution
xi
i
p(u)
du
=
n , i = 1, n.
−π
If
N ln N
n
→ 0 for n → ∞, then
EUnN = θ1 + O
2
(2N + 1)σnN
Z π
σ2
p−1 (u) du,
, θ1 =
(2π)2 −π
Z π
σ4
→ θ2 =
p−2 (u) du.
4π 3 −π
N ln N
n
(3)
(4)
Proof. From the definition of xi we easily obtain
1
1
1+O
,
∆i =
np(xi )
n
where O n1 is uniform with respect to i = 1, n.
Hence it follows that
Qir =
1
1
K
)
1
+
O
(x
−
x
.
N
r
i
n2 p(xi )p(xr )
n
(5)
Taking into account the relation
max |KN (u)| = O(N )
−π≤u≤π
(6)
and (5), we find
2
σnN
=
n
n X
1
X
1
σ4
2
(xi −xj )
KN
+O
.
2
2
2
2
2π (2N +1) n i=1 j=1
[p(xi )p(xj )]
n
(7)
Let F (x) be a distribution function with density p(x) and Fn (x) be an
empirical
Pn distribution function of the “sample” x1 , x2 , . . . , xn , i.e., Fn (x) =
n−1 k=1 I(−∞,x) (xk ), where IA (·) is the indicator of the set A. Then the
right side of (7) can be written as the integral
2
σnN
=
σ4
2π 2 (2N + 1)2
Z
π
−π
Z
π
−π
2
KN
(t − s)
1
dFn (t) dFn (s)
O
+
.
[p(t)p(s)]2
n
200
E. NADARAYA
Further we have
Z Z
Z
π π 2
dFn (t) dFn (s)
s)
−
(t
−
K
N
[p(t)p(s)]2
−π
−π
π
−π
Z
π
−π
2
(t − s)
KN
≤ I1 + I2 ,
dF (t) dF (s)
≤
[p(t)p(s)]2
Z π Z π
dFn (s)
2
I1 =
dFn (t) − dF (t) ,
KN (t − s)
[p(t)p(s)]2
−π −π
Z Z
π π 2
dF (t)
I2 =
KN (t − s)
dFn (s) − dF (s) .
[p(t)p(s)]2
−π −π
By integration by parts in the internal integral in I1 we readily obtain
Z
Z π
′
dFn (s) π
(t − s)p(t) −
dFn (t) − dF (t) KN
I1 ≤ 2
2
−π
−π p (s)
(8)
−KN (t − s)p′ (t) KN (t − s)/p3 (t) dt.
1
Since sup−π≤x≤π |Fn (x) − F (x)| = O n and the relations [8]1
Z π
2
′
KN
(u) du = 2N + 1,
(u)| = O(N 2 ),
max |KN
−π≤u≤π
−π
(9)
Z π
|KN (u)| du = O(ln N )
−π
are fulfilled, from (8) we have the estimate
2
N ln N
I1 = O
.
n
In the same manner we show that
I2 = O
Therefore
2
(2N +1)σnN
σ4
= 3
4π
Z
π
−π
Z
π
N 2 ln N
n
.
dt ds
ΦN (s−t)
+O
p(s)p(t)
−π
N ln N
n
,
(10)
2
where ΦN (u) = 2N2π+1 KN
(u) is the Fej´er kernel.
We shall complete the definition of the function p−1 outside [−π, π] as
regards its periodicity and also note that KN (u) and ΦN (u) are periodic
functions with the period 2π. The continued function will be denoted by
g(x). Then
Z π
Z π Z π
dt ds
p−2 (x) dx + χn ,
=
ΦN (s − t)
p(s)p(t)
−π
−π −π
1 See
p. 115 in the Russian version of [8]: “Mir”, Moscow, 1965.
LIMIT DISTRIBUTION OF THE SQUARED ERROR
201
where
|χn | ≤
Z
π
|¯
σN (x) − g(x)|dx,
−π
π
σ
¯N (x) =
Z
−π
ΦN (u)g(x − u)du.
Hence, on account of the theorem on convergence of the Fej´er integral
¯N (x) to g(x) in the norm of the space L1 (−π, π) (see [9], p.481), we have
σ
χn → 0 for n → ∞.
Therefore
Z π
σ4
2
p−2 (x) dx.
(2N + 1)σnN →
4π 3 −π
Now we shall prove (3). We have
DµnN (x) = σ
2
n
X
j=1
1
1
2
.
K (x − xj ) 1 + O
np2 (xj ) N
n
Applying the same reasoning as in deriving (10), we find
Z
N 2 ln N
σ2 π 2
ds
KN (x − s)
DµnN (x) =
+O
.
n −π
p(s)
n2
(11)
Therefore
Z
Z
N ln N
ds dt
+O
=
p(s)
n
−π −π
Z π
N ln N
σ2
=
.
p−1 (s) ds + O
2
(2π) −π
n
EUnN =
σ2
(2π)2
π
π
ΦN (t − s)
d
Denote by the symbol → the convergence in distribution, and let ξ be a
random variable having normal distribution with zero mean and variance 1.
2
Theorem 1. Let xi i = 1, n be the same as in Lemma 2 and N nln N → 0
√
−1/2 d
→ ξ.
for n → ∞. Then, as n increases, 2N + 1(UnN − θ1 )θ2
Proof. We have
UnN − EUnN
= Hn(1) + Hn(2) ,
σnN
where
Hn(1) =
n
X
j=1
ξj , Hn(2) =
n
X
n
(ε2 − Eε2i )Qii .
2π(2N + 1)σnN i=1 i
202
E. NADARAYA
(2)
Hn converges to zero in probability. Indeed,
n
X
n2 Eε14
2
Qii
=
2
(2π)2 (2N + 1)2 σnN
i=1
n
1
X
Eε41
1
2
K (0) 1 + O
≤
=
2 · n2
(2π)2 (2N + 1)2 σnN
(p(xi ))4 N
n
i=1
N
1
≤C 2 =O
,
nσnN
n
DHn(2) ≤
(2)
P
whence Hn → 0. Here and in what follows C is the positive constant
varying from one formula to another and the letter P above the arrow
denotes convergence in probability.
(1) d
We will now prove that Hn → ξ. To this end we will verify the validity
of Corollaries 2 and 6 of Theorem 2 from [6]. We have to show whether
the conditions contained in these statements are fulfilled for asymptotic
normality of the square-integrable martingale-difference, which, by Lemma
1, is our sequence {ξk , Fk }k≥1 .
Pn
A direct calculation shows that k=1 Eξk2 = 1. Asymptotic normality
will take place if for n → ∞
n
X
E ξk2 · I(|ξk | ≥ ε) | Fk−1 → 0
(12)
k=1
and
n
X
k=1
P
ξk2 → 1.
(13)
P
It is shown in [6] that the fulfillment of (13) and the condition sup |ξk | → 0
1≤k≤n
implies the validy of (12) as well.
Since for ε > 0
P
(1)
n
X
Eξk4 ,
sup |ξk | ≥ ε ≤ ε−4
1≤k≤n
k=1
d
to prove Hn → ξ we have to verify only (13) by the relation (15) to be
given below.
Pn
P
We will establish k=1 ξk2 → 1. For this itPsuffices to make sure that
P
n
n
2
2
E( k=1 ξk − 1)) → 0 for n → ∞, i.e., due to i=1 Eξi2 = 1
E
n
X
k=1
ξk2
2
=
n
X
k=1
Eξk4 + 2
X
1≤k1
1(1994), No. 2, 197-212
LIMIT DISTRIBUTION OF THE INTEGRATED SQUARED
ERROR OF TRIGONOMETRIC SERIES REGRESSION
ESTIMATOR
E. NADARAYA
Abstract. Limit distribution is studied for the integrated squared
error of the projection regression estimator (2) constructed on the
basis of independent observations (1). By means of the obtained
limit theorems, a test is given for verifying the hypothesis on the
regression, and the power of this test is calculated in the case of
Pitman alternatives.
Let observations Y1 , Y2 , . . . , Yn be represented as
Yi = µ(xi ) + εi ,
i = 1, n,
(1)
where µ(x), x ∈ [−π, π], is the unknown regression function to be estimated
by observations Yi ; xi , i = 1, n, are the known numbers, and −π = x0 <
x1 < · · · < xn ≤ π, εi , i = 1, n, are independent equidistributed random
variables; Eε1 = 0, Eε21 = σ 2 , and Eε41 < ∞.
The problem of nonparametric estimation of the regression function µ(x)
for the model (1) has a recent history and has been treated only in few
papers. In particular, a kernel estimator of the Rosenblatt–Parzen type for
µ(x) was proposed for the first time in [1].
Assume that µ(x) is representable as a converging series in L2 (−π, π)
with respect to the orthonormal trigonometric system
o∞
n
.
(2π)−1/2 , π −1/2 cos ix, π −1/2 sin ix
i=1
Consider the estimator of the function µ(x) constructed by the projection
method of N.N. Chentsov [2]
N
a0n X
µnN (x) =
ain cos ix + bin sin ix,
+
2
i=1
1991 Mathematics Subject Classification. 62G07.
197
(2)
198
E. NADARAYA
where N = N (n) → ∞ for n → ∞ and
ain =
n
n
1X
1X
Yj ∆j cos ixj , bin =
Yj ∆j sin ixj ,
π j=1
π j=1
∆j = xj − xj−1 , j = 1, n,
i = 0, N .
The estimator (2) can be rewritten in a more compact way as
µnN =
n
X
j=1
where KN (u) =
1
2π
P
Yj ∆j KN (x − xj ),
eiru is the Dirichlet kernel.
|r|≤N
In [3], p.347, N.V. Smirnov considered estimators of the type (2) for
a specially chosen class of functions µ(x) in the case of equidistant points
xj ∈ [−π, π] and of independent and normally distributed observation errors
εi . In [4] an estimator of the type (2) is obtained, which is asymptotically
equivalent to projection estimators which are optimal in the sense of some
accuracy criterion. The asymptotics of the mean value of the integrated
squared error of the estimator (2) is considered in [5].
It is of interest to investigate the limit distribution of the integrated
squared error
Z
π
−π
[µnN (x) − µ(x)]2 dx,
which is the goal pursued in this paper. The method used to prove the
theorems below is based on the functional limit theorem for a sequence of
semimartingales [6].
Denote
Z π
2
n
µnN (x) − EµnN (x) dx,
UnN =
2π(2N + 1) −π
2
Qir = ∆i ∆r KN (xi − xr ), σnN
=
ηik =
r−1
n X
X
n2 σ 4
Q2 ,
π 2 (2N + 1)2 r=2 j=1 jr
n
εi εk Qik ,
π(2N + 1)σnN
ξ1 = 0, ξk =
k−1
X
ηik ,
k = 2, n, ξk = 0, k > n,
i=1
and assume that Fk is σ-algebra generated by random variables ε1 , ε2 ,
. . . , εk , F0 = (φ, Ω).
Lemma 1 ([7], p.179). The stochastic sequence (ξk , Fk )k≥1 is a martingale-difference.
LIMIT DISTRIBUTION OF THE SQUARED ERROR
199
Lemma 2. Let p(x) be the known positive continuously differentiable
density on [−π, π], and points xi be chosen from the relation
Rdistribution
xi
i
p(u)
du
=
n , i = 1, n.
−π
If
N ln N
n
→ 0 for n → ∞, then
EUnN = θ1 + O
2
(2N + 1)σnN
Z π
σ2
p−1 (u) du,
, θ1 =
(2π)2 −π
Z π
σ4
→ θ2 =
p−2 (u) du.
4π 3 −π
N ln N
n
(3)
(4)
Proof. From the definition of xi we easily obtain
1
1
1+O
,
∆i =
np(xi )
n
where O n1 is uniform with respect to i = 1, n.
Hence it follows that
Qir =
1
1
K
)
1
+
O
(x
−
x
.
N
r
i
n2 p(xi )p(xr )
n
(5)
Taking into account the relation
max |KN (u)| = O(N )
−π≤u≤π
(6)
and (5), we find
2
σnN
=
n
n X
1
X
1
σ4
2
(xi −xj )
KN
+O
.
2
2
2
2
2π (2N +1) n i=1 j=1
[p(xi )p(xj )]
n
(7)
Let F (x) be a distribution function with density p(x) and Fn (x) be an
empirical
Pn distribution function of the “sample” x1 , x2 , . . . , xn , i.e., Fn (x) =
n−1 k=1 I(−∞,x) (xk ), where IA (·) is the indicator of the set A. Then the
right side of (7) can be written as the integral
2
σnN
=
σ4
2π 2 (2N + 1)2
Z
π
−π
Z
π
−π
2
KN
(t − s)
1
dFn (t) dFn (s)
O
+
.
[p(t)p(s)]2
n
200
E. NADARAYA
Further we have
Z Z
Z
π π 2
dFn (t) dFn (s)
s)
−
(t
−
K
N
[p(t)p(s)]2
−π
−π
π
−π
Z
π
−π
2
(t − s)
KN
≤ I1 + I2 ,
dF (t) dF (s)
≤
[p(t)p(s)]2
Z π Z π
dFn (s)
2
I1 =
dFn (t) − dF (t) ,
KN (t − s)
[p(t)p(s)]2
−π −π
Z Z
π π 2
dF (t)
I2 =
KN (t − s)
dFn (s) − dF (s) .
[p(t)p(s)]2
−π −π
By integration by parts in the internal integral in I1 we readily obtain
Z
Z π
′
dFn (s) π
(t − s)p(t) −
dFn (t) − dF (t) KN
I1 ≤ 2
2
−π
−π p (s)
(8)
−KN (t − s)p′ (t) KN (t − s)/p3 (t) dt.
1
Since sup−π≤x≤π |Fn (x) − F (x)| = O n and the relations [8]1
Z π
2
′
KN
(u) du = 2N + 1,
(u)| = O(N 2 ),
max |KN
−π≤u≤π
−π
(9)
Z π
|KN (u)| du = O(ln N )
−π
are fulfilled, from (8) we have the estimate
2
N ln N
I1 = O
.
n
In the same manner we show that
I2 = O
Therefore
2
(2N +1)σnN
σ4
= 3
4π
Z
π
−π
Z
π
N 2 ln N
n
.
dt ds
ΦN (s−t)
+O
p(s)p(t)
−π
N ln N
n
,
(10)
2
where ΦN (u) = 2N2π+1 KN
(u) is the Fej´er kernel.
We shall complete the definition of the function p−1 outside [−π, π] as
regards its periodicity and also note that KN (u) and ΦN (u) are periodic
functions with the period 2π. The continued function will be denoted by
g(x). Then
Z π
Z π Z π
dt ds
p−2 (x) dx + χn ,
=
ΦN (s − t)
p(s)p(t)
−π
−π −π
1 See
p. 115 in the Russian version of [8]: “Mir”, Moscow, 1965.
LIMIT DISTRIBUTION OF THE SQUARED ERROR
201
where
|χn | ≤
Z
π
|¯
σN (x) − g(x)|dx,
−π
π
σ
¯N (x) =
Z
−π
ΦN (u)g(x − u)du.
Hence, on account of the theorem on convergence of the Fej´er integral
¯N (x) to g(x) in the norm of the space L1 (−π, π) (see [9], p.481), we have
σ
χn → 0 for n → ∞.
Therefore
Z π
σ4
2
p−2 (x) dx.
(2N + 1)σnN →
4π 3 −π
Now we shall prove (3). We have
DµnN (x) = σ
2
n
X
j=1
1
1
2
.
K (x − xj ) 1 + O
np2 (xj ) N
n
Applying the same reasoning as in deriving (10), we find
Z
N 2 ln N
σ2 π 2
ds
KN (x − s)
DµnN (x) =
+O
.
n −π
p(s)
n2
(11)
Therefore
Z
Z
N ln N
ds dt
+O
=
p(s)
n
−π −π
Z π
N ln N
σ2
=
.
p−1 (s) ds + O
2
(2π) −π
n
EUnN =
σ2
(2π)2
π
π
ΦN (t − s)
d
Denote by the symbol → the convergence in distribution, and let ξ be a
random variable having normal distribution with zero mean and variance 1.
2
Theorem 1. Let xi i = 1, n be the same as in Lemma 2 and N nln N → 0
√
−1/2 d
→ ξ.
for n → ∞. Then, as n increases, 2N + 1(UnN − θ1 )θ2
Proof. We have
UnN − EUnN
= Hn(1) + Hn(2) ,
σnN
where
Hn(1) =
n
X
j=1
ξj , Hn(2) =
n
X
n
(ε2 − Eε2i )Qii .
2π(2N + 1)σnN i=1 i
202
E. NADARAYA
(2)
Hn converges to zero in probability. Indeed,
n
X
n2 Eε14
2
Qii
=
2
(2π)2 (2N + 1)2 σnN
i=1
n
1
X
Eε41
1
2
K (0) 1 + O
≤
=
2 · n2
(2π)2 (2N + 1)2 σnN
(p(xi ))4 N
n
i=1
N
1
≤C 2 =O
,
nσnN
n
DHn(2) ≤
(2)
P
whence Hn → 0. Here and in what follows C is the positive constant
varying from one formula to another and the letter P above the arrow
denotes convergence in probability.
(1) d
We will now prove that Hn → ξ. To this end we will verify the validity
of Corollaries 2 and 6 of Theorem 2 from [6]. We have to show whether
the conditions contained in these statements are fulfilled for asymptotic
normality of the square-integrable martingale-difference, which, by Lemma
1, is our sequence {ξk , Fk }k≥1 .
Pn
A direct calculation shows that k=1 Eξk2 = 1. Asymptotic normality
will take place if for n → ∞
n
X
E ξk2 · I(|ξk | ≥ ε) | Fk−1 → 0
(12)
k=1
and
n
X
k=1
P
ξk2 → 1.
(13)
P
It is shown in [6] that the fulfillment of (13) and the condition sup |ξk | → 0
1≤k≤n
implies the validy of (12) as well.
Since for ε > 0
P
(1)
n
X
Eξk4 ,
sup |ξk | ≥ ε ≤ ε−4
1≤k≤n
k=1
d
to prove Hn → ξ we have to verify only (13) by the relation (15) to be
given below.
Pn
P
We will establish k=1 ξk2 → 1. For this itPsuffices to make sure that
P
n
n
2
2
E( k=1 ξk − 1)) → 0 for n → ∞, i.e., due to i=1 Eξi2 = 1
E
n
X
k=1
ξk2
2
=
n
X
k=1
Eξk4 + 2
X
1≤k1