Random Effects Models

13.11 Random Effects Models

Throughout this chapter, we deal with analysis-of-variance procedures in which the primary goal is to study the effect on some response of certain fixed or prede- termined treatments. Experiments in which the treatments or treatment levels are preselected by the experimenter as opposed to being chosen randomly are called fixed effects experiments. For the fixed effects model, inferences are made only on those particular treatments used in the experiment.

It is often important that the experimenter be able to draw inferences about

a population of treatments by means of an experiment in which the treatments used are chosen randomly from the population. For example, a biologist may

be interested in whether or not there is significant variance in some physiological characteristic due to animal type. The animal types actually used in the experiment are then chosen randomly and represent the treatment effects. A chemist may be interested in studying the effect of analytical laboratories on the chemical analysis of a substance. She is not concerned with particular laboratories but rather with

a large population of laboratories. She might then select a group of laboratories

548 Chapter 13 One-Factor Experiments: General at random and allocate samples to each for analysis. The statistical inference

would then involve (1) testing whether or not the laboratories contribute a nonzero variance to the analytical results and (2) estimating the variance due to laboratories and the variance within laboratories.

Model and Assumptions for Random Effects Model

The one-way random effects model is written like the fixed effects model but with the terms taking on different meanings. The response y ij =μ+α i +ǫ ij is now a value of the random variable

Y ij =μ+A i +ǫ ij , with i = 1, 2, . . . , k and j = 1, 2, . . . , n, where the A i are independently and normally distributed with mean 0 and variance

σ 2 α and are independent of the ǫ ij . As for the fixed effects model, the ǫ ij are also independently and normally distributed with mean 0 and variance σ 2 . Note that

for a random effects experiment, the constraint that α i = 0 no longer applies.

i=1

Theorem 13.4: For the one-way random effects analysis-of-variance model,

2 2 E(SSA) = (k − 1)σ 2 + n(k − 1)σ

E(SSE) = k(n − 1)σ . Table 13.11 shows the expected mean squares for both a fixed effects and a

and

random effects experiment. The computations for a random effects experiment are carried out in exactly the same way as for a fixed effects experiment. That is, the sum-of-squares, degrees-of-freedom, and mean-square columns in an analysis- of-variance table are the same for both models.

Table 13.11: Expected Mean Squares for the One-Factor Experiment Source of

Expected Mean Squares Variation

Degrees of

Mean

Freedom

Squares Fixed Effects

Random Effects

s 2 σ 2 σ 2 Total

For the random effects model, the hypothesis that the treatment effects are all zero is written as follows:

Hypothesis for a

H :σ Random Effects 2 0 α = 0,

H :σ Experiment 2 1 α

This hypothesis says that the different treatments contribute nothing to the variability of the response. It is obvious from Table 13.11 that s 2 1 and s 2 are both

13.11 Random Effects Models 549

estimates of σ 2 when H 0 is true and that the ratio s 2

f= 1 s 2

is a value of the random variable F having the F-distribution with k−1 and k(n−1) degrees of freedom. The null hypothesis is rejected at the α-level of significance when

f>f α [k − 1, k(n − 1)].

In many scientific and engineering studies, interest is not centered on the F- test. The scientist knows that the random effect does, indeed, have a significant effect. What is more important is estimation of the various variance components. This produces a ranking in terms of what factors produce the most variability and by how much. In the present context, it may be of interest to quantify how much larger the single-factor variance component is than that produced by chance (random variation).

Estimation of Variance Components

Table 13.11 can also be used to estimate the variance components σ 2 and σ 2 .

Since s 2 estimates σ 2 + nσ 2

1 α and s 2 estimates σ 2 ,

2 2 2 s 1 −s σ 2 ˆ =s , σ ˆ

Example 13.7: The data in Table 13.12 are coded observations on the yield of a chemical process, using five batches of raw material selected randomly. Show that the batch variance component is significantly greater than zero and obtain its estimate.

Table 13.12: Data for Example 13.7

55.6 59.7 80.7 58.4 75.3 329.7 Solution : The total, batch, and error sums of squares are, respectively,

Total

SST = 194.64, SSA = 72.60, and SSE = 194.64 − 72.60 = 122.04. These results, with the remaining computations, are shown in Table 13.13.

550 Chapter 13 One-Factor Experiments: General

Table 13.13: Analysis of Variance for Example 13.7 Source of

Sum of

Degrees of

The f-ratio is significant at the α = 0.05 level, indicating that the hypothesis of

a zero batch component is rejected. An estimate of the batch variance component is

Note that while the batch variance component is significantly different from zero, when gauged against the estimate of σ 2 , namely ˆ σ 2 = M SE = 4.07, it appears as if the batch variance component is not appreciably large. If the result using the formula for σ 2 α appears negative, (i.e., when s 2 1 is smaller than s 2 ), ˆ σ 2 α is then set to zero. This is a biased estimator. In order to have

a better estimator of σ 2 α , a method called restricted (or residual) maximum likelihood (REML) is commonly used (see Harville, 1977, in the Bibliography). Such an estimator can be found in many statistical software packages. The details for this estimation procedure are beyond the scope of this text.

Randomized Block Design with Random Blocks

In a randomized complete block experiment where the blocks represent days, it is conceivable that the experimenter would like the results to apply not only to the actual days used in the analysis but to every day in the year. He or she would then select at random the days on which to run the experiment as well as the treatments and use the random effects model

Y ij =μ+A i +B j +ǫ ij , for i = 1, 2, . . . , k and j = 1, 2, . . . , b, with the A i ,B j , and ǫ ij being independent random variables with means 0 and

variances σ 2 2 α 2 ,σ β , and σ , respectively. The expected mean squares for a random effects randomized complete block design are obtained, using the same procedure as for the one-factor problem, and are presented along with those for a fixed effects experiment in Table 13.14.

Again the computations for the individual sums of squares and degrees of free- dom are identical to those of the fixed effects model. The hypothesis

H 0 :σ 2 α = 0,

H 1 :σ 2 α

is carried out by computing

s 2 f= 1

13.12 Case Study 551

Table 13.14: Expected Mean Squares for the Randomized Complete Block Design Source of

Expected Mean Squares Variation

Degrees of

Mean

Fixed Effects Random Effects Treatments

s 2 σ 2 σ 2 Total

Error

(k − 1)(b − 1)

kb − 1 and rejecting H 0 when f > f α [k − 1, (b − 1)(k − 1)].

The unbiased estimates of the variance components are

Tests of hypotheses concerning the various variance components are made by computing the ratios of appropriate mean squares, as indicated in Table 13.14, and comparing them with corresponding f-values from Table A.6.