Methods of Point Estimation
6.2 Methods of Point Estimation
We now introduce two “constructive” methods for obtaining point estimators: the method of moments and the method of maximum likelihood. By constructive we mean that the general definition of each type of estimator suggests explicitly how to obtain the estimator in any specific problem. Although maximum likelihood esti mators are generally preferable to moment estimators because of certain efficiency properties, they often require significantly more computation than do moment esti mators. It is sometimes the case that these methods yield unbiased estimators.
the Method of Moments
The basic idea of this method is to equate certain sample characteristics, such as the mean, to the corresponding population expected values. Then solving these equa tions for unknown parameter values yields the estimators.
DEFINITION Let X 1 ,…, X n
be a random sample from a pmf or pdf f(x). For k 5 1, 2,
3,…, the kth population moment, or kth moment of the distribution f(x), is E(X k ). The kth sample moment is (1
i5 1 X yn)o k i .
n
6.2 Methods of point estimation 265
Thus the first population moment is E(X) 5 m, and the first sample moment
is oX i yn 5 X. The second population and sample moments are E(X 2 ) and
oX 2 i yn, respectively. The population moments will be functions of any unknown
parameters u 1 ,u 2 , ….
DEFINITION Let X 1 , X 2 ,…, X n
be a random sample from a distribution with pmf or pdf
f (x; u 1 ,…, u m ), where u 1 ,…, u m are parameters whose values are unknown. Then the moment estimators uˆ 1 ,…,uˆ m are obtained by equating the first m sample
moments to the corresponding first m population moments and solving for
u 1 ,…, u m .
If, for example, m 5 2, E(X) and E(X 2 ) will be functions of u 1 and u 2 . Setting
E (X) 5 (1 yn)oX i (5 X) and E(X 2 ) 5 (1
yn)oX 2
i gives two equations in u 1 and u 2 .
The solution then defines the estimators.
ExamplE 6.12
Let X 1 , X 2 ,…, X n represent a random sample of service times of n customers at
a certain facility, where the underlying distribution is assumed exponential with parameter l. Since there is only one parameter to be estimated, the estimator is obtained by equating E(X ) to X. Since E(X ) 5 1 yl for an exponential distribution, this gives 1 yl 5 X or l 5 1yX. The moment estimator of l is then lˆ 5 1yX.
be a random sample from a gamma distribution with parameters a and
b . From Section 4.4, E(X ) 5 ab and E(X 2 )5b 2 G (a 1 2) yG(a) 5 b 2 (a 1 1)a.
The moment estimators of a and b are obtained by solving
n o i
X 5 ab X 2
5a (a 1 1)b 2
Since a(a 1 1)b 2
5a 2 2 b 2 1 ab 2 and the first equation implies a 2 b 2 5X , the sec
ond equation becomes
i 5 X 2 1 ab o 2
X 2
n
Now dividing each side of this second equation by the corresponding side of the first equation and substituting back gives the estimators
(1 yn) X 2 X i 2 X
o i
2 X 2 2 (1 yn) X X
aˆ 5
o
bˆ 5
To illustrate, the survivaltime data mentioned in Example 4.24 is 152 115 109 94 88 137 152 77 160 165
125 40 128 123 136 101 62 153 83 69 from which x 5 113.5 and (1
y20)ox 2
i 5 14,087.8. The parameter estimates are (113.5) 2 14,087.8 2 (113.5) 2
These estimates of a and b differ from the values suggested by Gross and Clark because they used a different estimation technique.
n
266 Chapter 6 point estimation
ExamplE 6.14
Let X 1 ,…, X n
be a random sample from a generalized negative binomial
distribution with parameters r and p (see Section 3.5). Since E(X) 5 r(1 2 p) yp
and V(X ) 5 r (1 2 p) yp 2 , E(X 2 ) 5 V(X ) 1 [E(X )] 2 5 r (1 2 p)(r 2 rp 1 1) yp 2 .
Equating E(X ) to X and E(X 2 ) to (1 yn)oX i 2 eventually gives
(1 yn) X 2 i 2 X (1 yn) X 2 X 2 X
X X 2
pˆ 5
o
rˆ 5
o i
As an illustration, Reep, Pollard, and Benjamin (“Skill and Chance in Ball
Games,” J. of Royal Stat. Soc., 1971: 623–629) consider the negative binomial dis
tribution as a model for the number of goals per game scored by National Hockey League teams. The data for 1966–1967 follows (420 games):
x5 x o i y420 5 [(0)(29) 1 (1)(71) 1 . . . 1 (10)(3)]y420 5 2.98
and
x 2 2 (29) 1 (1) 2 (71) 1 . . . 1 (10) 2 o (3)] i y420 5 [(0) y420 5 12.40
Although r by definition must be positive, the denominator of rˆ could be negative, indicating that the negative binomial distribution is not appropriate (or that the moment estimator is flawed).
n