Linear Regression Model Using Matrices

12.3 Linear Regression Model Using Matrices

In fitting a multiple linear regression model, particularly when the number of vari- ables exceeds two, a knowledge of matrix theory can facilitate the mathematical manipulations considerably. Suppose that the experimenter has k independent

448 Chapter 12 Multiple Linear Regression and Certain Nonlinear Regression Models variables x 1 ,x 2 ,...,x k and n observations y 1 ,y 2 ,...,y n , each of which can be ex-

pressed by the equation y i =β 0 +β 1 x 1i +β 2 x 2i +···+β k x ki +ǫ i . This model essentially represents n equations describing how the response values

are generated in the scientific process. Using matrix notation, we can write the following equation:

General Linear

y = Xβ + ǫ,

Model where

11 21 ···x k1

y= ⎢ ⎥

12 22 ···x k2

β k ǫ n Then the least squares method for estimation of β, illustrated in Section 12.2,

1 x 1n x 2n ···x kn

involves finding b for which

SSE = (y − Xb) ′ (y − Xb)

is minimized. This minimization process involves solving for b in the equation

∂ (SSE) = 0. ∂b

We will not present the details regarding solution of the equations above. The result reduces to the solution of b in

(X ′ X)b = X ′ y.

Notice the nature of the X matrix. Apart from the initial element, the ith row represents the x-values that give rise to the response y i . Writing

allows the normal equations to be put in the matrix form

Ab = g.

12.3 Linear Regression Model Using Matrices 449 If the matrix A is nonsingular, we can write the solution for the regression

coefficients as

b=A −1 g = (X ′ X) −1 X ′ y.

Thus, we can obtain the prediction equation or regression equation by solving a set of k + 1 equations in a like number of unknowns. This involves the inversion of the k + 1 by k + 1 matrix X ′

X. Techniques for inverting this matrix are explained in most textbooks on elementary determinants and matrices. Of course, there are many high-speed computer packages available for multiple regression problems, packages that not only print out estimates of the regression coefficients but also provide other information relevant to making inferences concerning the regression equation.

Example 12.4: The percent survival rate of sperm in a certain type of animal semen, after storage, was measured at various combinations of concentrations of three materials used to increase chance of survival. The data are given in Table 12.3. Estimate the multiple linear regression model for the given data.

Table 12.3: Data for Example 12.4

y (% survival) x 1 (weight %) x 2 (weight %) x 3 (weight %)

26.5 1.70 5.30 8.20 Solution : The least squares estimating equations, (X ′ X)b = X ′ y, are

b 3 3337.780 From a computer readout we obtain the elements of the inverse matrix

(X X) = ⎢ −0.0826

0.0886 and then, using the relation b = (X ′ X) −1 X ′ y, the estimated regression coefficients are obtained as

450 Chapter 12 Multiple Linear Regression and Certain Nonlinear Regression Models

b 0 = 39.1574, b 1 = 1.0161, b 2 = −1.8616, b 3 = −0.3433. Hence, our estimated regression equation is

ˆ y = 39.1574 + 1.0161x 1 − 1.8616x 2 − 0.3433x 3 .

Exercises

Test Classes mine a way of predicting cooking time y at various

12.1 A set of experimental runs was made to deter-

Chemistry

Student Grade, y Score, x 1 Missed, x 2

1 85 65 1 coded data were recorded as follows:

values of oven width x 1 and flue temperature x 2 . The

2 74 50 7 y

x 1 x 2 3 76 55 5 6.40 1.32 1.15 4 90 65 2

10.65 40.40 (a) Fit a multiple linear regression equation of the form Estimate the multiple linear regression equation

y=b ˆ 0 +b 1 x 1 +b 2 x 2 .

μ Y |x 1 ,x 2 =β 0 +β 1 x 1 +β 2 x 2 .

(b) Estimate the chemistry grade for a student who has an intelligence test score of 60 and missed 4 classes.

12.4 An experiment was conducted to determine if 12.2 In Applied Spectroscopy, the infrared reflectance the weight of an animal can be predicted after a given spectra properties of a viscous liquid used in the elec- period of time on the basis of the initial weight of the tronics industry as a lubricant were studied. The de- animal and the amount of feed that was eaten. The signed experiment consisted of the effect of band fre- following data, measured in kilograms, were recorded:

quency x 1 and film thickness x 2 on optical density y

using a Perkin-Elmer Model 621 infrared spectrometer.

Initial Feed (Source: Pacansky, J., England, C. D., and Wattman,

Final

Weight, x 1 Weight, x 2 R., 1986.)

Weight, y

0.31 (a) Fit a multiple regression equation of the form 2.948

1.10 μ Y |x 1 ,x 2 =β 0 +β 1 x 1 +β 2 x 2 . 1.633

0.31 (b) Predict the final weight of an animal having an ini- Estimate the multiple linear regression equation

tial weight of 35 kilograms that is given 250 kilo- grams of feed.

y=b ˆ 0 +b 1 x 1 +b 2 x 2 .

12.5 The electric power consumed each month by a chemical plant is thought to be related to the average 12.3 Suppose in Review Exercise 11.53 on page 437 ambient temperature x 1 , the number of days in the that we were also given the number of class periods month x 2 , the average product purity x 3 , and the tons missed by the 12 students taking the chemistry course. of product produced x 4 . The past year’s historical data The complete data are shown.

are available and are presented in the following table.

Exercises 451 y

(a) Fit a multiple linear regression model using the above data set.

(b) Predict power consumption for a month in which x 1 = 75 ◦

2 F, x = 24 days, x 3 = 90%, and x 4 = 98

tons. 12.6 An experiment was conducted on a new model

of a particular make of automobile to determine the stopping distance at various speeds. The following data were recorded.

Speed, v (km/hr)

Stopping Distance, d (m) 16 26 41 62 88 119 (a) Fit a multiple regression curve of the form μ D|v =

β 0 +β 1 v+β 2 v 2 . (b) Estimate the stopping distance when the car is

traveling at 70 kilometers per hour. 12.7 An experiment was conducted in order to de-

termine if cerebral blood flow in human beings can be predicted from arterial oxygen tension (millimeters of mercury). Fifteen patients participated in the study, and the following data were collected:

Blood Flow,

Arterial Oxygen

Tension, x

Estimate the quadratic regression equation

μ Y |x =β 0 +β 1 x+β 2 x 2 .

12.8 The following is a set of coded experimental data on the compressive strength of a particular alloy at var- ious values of the concentration of some additive:

Concentration,

Compressive

Strength, y

(a) Estimate the quadratic regression equation μ Y |x =

β 0 +β 1 x+β 2 x 2 .

(b) Test for lack of fit of the model. 12.9 (a) Fit a multiple regression equation of the

form μ Y |x =β 0 +β 1 x 1 +β 2 x 2 to the data of Ex-

ample 11.8 on page 420. (b) Estimate the yield of the chemical reaction for a

temperature of 225 ◦ C.

12.10 The following data are given:

1 4 5 3 2 3 4 (a) Fit the cubic model μ Y |x =β 0 +β 1 x+β 2 x 2 +β 3 x 3 .

(b) Predict Y when x = 2.

12.11 An experiment was conducted to study the size of squid eaten by sharks and tuna. The regressor vari- ables are characteristics of the beaks of the squid. The data are given as follows:

452 Chapter 12 Multiple Linear Regression and Certain Nonlinear Regression Models In the study, the regressor variables and response con- were obtained. (From Response Surface Methodology,

sidered are Myers, Montgomery, and Anderson-Cook, 2009.) x 1 = rostral length, in inches,

x 1 x 2 y x 1 x 2 x 2 = wing length, in inches,

230 15.5 816 x 3 = rostral to notch length, in inches,

91 43.0 1201 x 4 = notch to wing length, in inches,

125 40.0 1115 x 5 = width, in inches,

(a) Estimate the unknown parameters of the multiple y = weight, in pounds.

linear regression equation

Estimate the multiple linear regression equation μ Y |x 1 ,x 2 =β 0 +β 1 x 1 +β 2 x 2 . μ Y |x 1 ,x 2 ,x 3 ,x 4 ,x 5

=β 0 +β 1 x 1 +β 2 x 2 +β 3 x 3 +β 4 x 4 +β 5 x 5 .

(b) Predict wear when oil viscosity is 20 and load is

12.12 The following data reflect information from 17 U.S. Naval hospitals at various sites around the world.

12.14 Eleven student teachers took part in an eval- The regressors are workload variables, that is, items uation program designed to measure teacher effective- that result in the need for personnel in a hospital. A ness and determine what factors are important. The brief description of the variables is as follows:

response measure was a quantitative evaluation of the y = monthly labor-hours,

teacher. The regressor variables were scores on four standardized tests given to each teacher. The data are

x 1 = average daily patient load,

as follows:

x 2 = monthly X-ray exposures,

x 1 x 2 x 3 x 4 x 3 = monthly occupied bed-days,

59.00 55.66 x 4 = eligible population in the area/1000,

31.75 63.97 x 5 = average length of patient’s stay, in days.

Site x 1 x 2 x 3 x 4 x 5 y

1687.00 43.3 5.62 1854.17 Estimate the multiple linear regression equation 8 59.28 5969

μ Y |x 1 ,x 2 ,x 3 ,x 4 =β 0 +β 1 x 1 +β 2 x 2 +β 3 x 3 +β 4 x 4 . 10 128.02 20,106

12.15 The personnel department of a certain indus- 13 127.21 15,543

3865.67 126.8 5.50 4026.52 trial firm used 12 subjects in a study to determine the 14 252.90 36,194

7684.10 157.7 7.00 10,343.81 relationship between job performance rating (y) and 15 409.20 34,703 12,446.33 169.4 10.75 11,732.17 scores on four tests. The data are as follows: 16 463.70 39,204 14,098.40 331.4 7.05 15,414.94

xx

11.2 56.5 71.0 38.5 43.0 The goal here is to produce an empirical equation that

14.5 59.5 72.5 38.2 44.8 will estimate (or predict) personnel needs for Naval

17.2 69.2 76.0 42.5 49.0 hospitals. Estimate the multiple linear regression equa-

17.8 74.5 79.5 43.4 56.3 tion

19.3 81.2 84.0 47.5 60.2 μ Y |x 1 ,x 2 ,x 3 ,x 4 ,x 5 24.5 88.0 86.2 47.4 62.0

=β 0 +β 1 x 1 +β 2 x 2 +β 3 x 3 +β 4 x 4 +β 5 x 5 .

12.13 A study was performed on a type of bear- 20.0 80.5 85.0 48.1 60.3 ing to find the relationship of amount of wear y to

x 1 = oil viscosity and x 2 = load. The following data

12.4 Properties of the Least Squares Estimators 453 Estimate the regression coefficients in the model

Emitter-RS

Base-RS

E-B-RS hFE

5.000 82.68 12.16 An engineer at a semiconductor company

6.625 112.60 wants to model the relationship between the gain or

5.750 97.52 hFE of a device (y) and three parameters: emitter-RS

6.125 111.80 data are shown below:

(x 1 ), base-RS (x 2 ), and emitter-to-base-RS (x 3 ). The

98.01 (Data from Myers, Montgomery, and Anderson-Cook, 14.50 226.5

(a) Fit a multiple linear regression to the data.

(cont.)

(b) Predict hFE when x 1 = 14, x 2 = 220, and x 3 = 5.