More Advanced Designs

7.5 More Advanced Designs

The lessR ANOVA function can analyze the three designs previously illustrated in this chapter: one-way ANOVA, randomized blocks ANOVA, and two-way ANOVA. These designs and subsequent data analysis are among the most common in the analysis of experimental data, but many other designs are encountered, some of them considerably more complex, including unbalanced designs. Kirk (2013) comprehensively categorizes, explains and illustrates these more complex designs.

To analyze data from designs more complex than the three previously presented in this chapter requires direct use of the standard R functions upon which lessR relies. Such analysis is beyond the scope of this text. One excellent reference in the application of R to more advanced designs is Faraway (2004). For balanced designs the relevant R function is aov . For unbalanced lme4 package: and even more sophisticated designs, there is the function lmer in the lme4 package.

Sophisticated functions for the

When using R functions directly, two rules must be followed. First, unlike the lessR analysis of data versions, the standard R functions make no default assumption regarding the data frame to from unbalanced and complex

be analyzed, which for lessR is mydata . For the aov function, specify the data frame with the designs. same data argument as with the lessR functions, such as data=mydata . Second, a complete analysis usually involves several to many R functions, and perhaps subsequent programming as well. The primary analysis of variance R function for balanced designs, aov , only specifies the initial analysis. Individual components of the analysis, such as the ANOVA summary table, must be explicitly referenced, here with the R function summary . By contrast the lessR functions weave these needed functions together and provide any needed ancillary programming. The lessR result is a more complete output from a single function call within the interactive R environment.

7.5.1 Randomized Block Factorial Design

randomized

The first more advanced design considered here is what Kirk (2013), p. 459, calls a block factorial design: Two or randomized block factorial design, also referred to as a within-samples or within-subjects design. more within

The randomized block design already presented yields a single data value for each block at each (dependent)

groups design.

treatment level of the one treatment variable. The randomized block factorial design applies the same concept to more than a single treatment variable. In the factorial version, each block yields

a single data value for each combination of levels of the treatment variables. As with the randomized block design, the data values for the block can consist of repeated measures of the same experimental unit such as a person, rat, organization, thing, place, or event. Or, the block can consist of different units matched on some relevant characteristic. This example applies the latter approach with a re-consideration of the previously presented two-way between-subjects factorial design and corresponding analysis of variance.

two-way ANOVA,

The previous example of the two-way ANOVA examined the effect on task completion Time Section 7.4 , p. 166 in response to one of three levels of Dosage of an arousal producing drug, and one of two levels of task Difficulty. The experimental subjects are laboratory rats. The task is running through a maze to obtain a food reward. The design is referred to as between-samples or between-groups or between-subjects design because each measurement of completion Time is provided by a

different rat in one cell, one of 3 × 2 = 6 unique combinations of Dosage and Difficulty. The study has 8 replications per cell, that is, 8 different rats who are exposed to the same treatment conditions for each of the 6 cells in the design.

174 Compare Multiple Samples

The randomized blocks factorial version of this study partitions the 48 rats into 8 groups of

6 based on an initial assessment of each rat’s ability to navigate a maze. That is, some rats in general do better than others. A trial maze served as a sort of a pre-test in which the rats were sorted on the basis of their ability to solve the maze. The first block of 6 rats ran the trial maze the fastest, and the last block the slowest. Within each block the rats were randomly assigned to each of the 6 treatment combinations. There are still 48 different rats in the study, but now each block of matched rats provides a score on each of the 6 treatment combinations. This design is referred to as within-subjects because similar rats in terms of maze running ability provide the data for each block of data values. Each rat in this block only experiences one of the 6 cells, but all the rats in a block are evaluated across all 6 combinations of the levels of the two treatment variables.

The first task is to read the data stored as a csv file in the long form with one data value per line in the data file.

> mydata <- Read("http://lessRstats.com/data/anova_rbf.csv")

Read function, Section 2.2.1 , p. 32

The first and last rows of this data table appear in Listing 7.24 . The data are the same data as from the two-way ANOVA, but here there is an additional variable, Block, with values that range from Blck1 to Blck8 . Each block contains 6 data values, one for each combination of levels of the two treatment variables. The data values in Listing 7.24 illustrate the first 6 data values for the first block.

> mydata Difficulty Dosage Block Time

1 Easy

mg00 Blck1 25.6

2 Easy

mg05 Blck1 23.4

3 Easy

mg10 Blck1 24.2

4 Hard

mg00 Blck1 40.5

5 Hard

mg05 Blck1 40.3

6 Hard

mg10 Blck1 46.7

7 Easy

mg00 Blck2 25.8

47 Hard

mg05 Blck8 27.4

48 Hard

mg10 Blck8 43.0

Listing 7.24 Subset of data for randomized block factorial design.

The analysis proceeds from the R function aov .

R Input Two treatment randomized block factorial design > fit <- aov(Time ∼ (Dosage*Difficulty) +

Error(Block/(Dosage*Difficulty)), data=mydata)

> summary(fit)

Note the similarity between the specification of the randomized block factorial design with aov and the two-way factorial design with ANOVA . The specifications are the same except for the addition of the Error term in the randomized block design. Both designs incorporate a

Compare Multiple Samples 175

complete factorial structure. In this example all 6 treatment combinations of 3 levels of Dosage and 2 Difficulty levels, with task completion Time as the response variable. Specify this aspect of the design with the following component.

Time ∼ (Dosage*Difficulty)

The distinction between the designs is the blocking structure of the randomized block factorial design, indicated with the additional following Error term, which allows for the customization of the error terms in the analysis. Each error term in the analysis of variance summary table is labeled Residuals.

Error(Block/(Dosage*Difficulty))

This notation indicates that there are four error terms in the analysis. The first term is differences among the blocks, indicated by Block. The remaining three terms are for Dosage, Difficulty, and their interaction, all within blocks.

The standard R approach to linear models such as analysis of variance is to estimate the model and then store the results into an object named something such as fit . Then different functions are applied to fit to obtain different analyses of the model. The summary function provides the core analysis, the ANOVA summary table, shown in Listing 7.25 .

Error: Block Df Sum Sq Mean Sq F value Pr(>F) Residuals 7 316.6

Error: Block:Dosage Df Sum Sq Mean Sq F value Pr(>F) Dosage

Error: Block:Difficulty Df Sum Sq Mean Sq F value

Pr(>F)

Error: Block:Dosage:Difficulty

Df Sum Sq Mean Sq F value

Pr(>F)

Dosage:Difficulty 2 402.6 201.30

Listing 7.25 Randomized blocks factorial design summary table from R.

The primary feature of this randomized blocks factorial design compared to the two-way ANOVA is the same advantage for the one treatment variable randomized blocks design. The consideration of blocks of similar participants allows for the partitioning of the associated sum of squares, which are then removed from the sum of squares error term. The result is a more powerful test, more able to detect existing differences between population means, if there are differences among the blocks.

176 Compare Multiple Samples

The primary result from the two-way ANOVA remains. The interaction of Dosage of the arousal drug and task Difficulty is significant.

Dosage x Difficulty Effect: p -value = 0 . 0002 <α= 0 . 05 , reject H 0

cell means plot example,

The meaning of this interaction was previously explored in the cell means plot for these

Section 7.19,

p. 169

data. Similarly, task Difficulty is significant, and Dosage is not significant. Difficulty Effect: p -value = 0 . 0000 <α= 0 . 05 , reject H 0

Dosage Effect: p -value = 0 . 0816 >α= 0 . 05 , do not reject H 0

This R analysis was specified to evaluate each of the three effects, two main effects and the interaction, with its own error term, which is the interaction of Block with the corresponding effect. Some authors, such as Kirk (2013), p. 462, combine the residual terms for each of the three effects, to serve as a single error baseline from which to evaluate each of the three effects. The logic for this combined term is that each error term only reflects random error, so their combination is a single indicator of the extent of random error in the analysis.

To obtain an analysis with this combined error term, specify a new, simpler error term in the call to aov .

> fit <- aov(Time ∼ (Dosage*Difficulty) + Error(Block), data=mydata) The output from summary(fit) is given in Listing 7.26 . The R output is slightly modified

in that the p-values are reported only to four significant digits.

Error: Block

Df Sum Sq Mean Sq F value Pr(>F)

Residuals 7 316.6

Error: Within

Df Sum Sq Mean Sq

F value Pr(>F)

Dosage:Difficulty 2 402.6

Listing 7.26 Analysis of the randomized block factorial design with a single within-subjects error term direct from the R function aov.

The Residuals degrees of freedom and sum of squares are just the sum of the corresponding three Residuals terms from Listing 7.25 . The revised Error specification in the call to aov specifies only a Block error term. The remaining sources of error variation that are “left over” are now combined into a generic Within error term, which refers to the within-subjects variation.

split-plot

factorial design:

7.5.2 Split-plot Factorial Design

One between- groups treatment

A second more advanced design is the split-plot factorial design. The simplest such design is

variable and one within-groups

illustrated here, with two treatment variables. One treatment variable is between-groups, that

treatment variable.

is, with different sets of unmatched participants. The other treatment variable is within-groups,

Compare Multiple Samples 177

that is, the defining feature of a randomized blocks design. Each participant in the study forms

a block of matched scores on the response variable across this within-groups treatment variable. one-way

The two-way split-plot factorial can be thought of as a combination of the one-way between- between-groups design, groups design and the randomized blocks design. The randomized blocks design has a data Section 7.2 , p. 150 value for each block of responses for each level of the treatment variable. The split-plot factorial replicates this structure with a second treatment condition for the between-groups treatment variable.

randomized blocks

To illustrate, return to the randomized-blocks design already presented. Each of 7 partici- design, Section 7.3 , p. 158 pants took one of four pre-workout supplements and then bench pressed 125 lbs for as many repetitions as possible. The design is randomized-blocks because each participant did this for each Supplement, a total of four times, which generated 28 data values. Supplement is a within- groups treatment variable because all four of its levels were administered to each participant. To generalize this design to the split-plot factorial, presume that all the participants in this previous study also had a controlled, highly nutritious breakfast exactly two hours before taking the pre-workout Supplement.

14 participants were recruited for the study and then each participant was randomly assigned to one of two groups. Suppose the second group of 7 participants also followed the same randomized-blocks design. Each participant also took all four Supplements, one for each workout, in a randomized order, again resulting in a total of 28 data values. The distinction is that for this second group their breakfast was less nutritious. Type of Supplements is a within-groups treatment variable, but now the other treatment variable, Food quality, is a between-groups treatment variable. The participants in the Low Food quality group are different participants from those in the Hi Food quality group.

Now suppose that actually 7 × 2 =

The first task is to read the data stored as a csv file in the long form with one data value

per line in the data file, resulting in 7 × 4 × 2 = 56 rows of data.

> mydata <- Read("http://lessRstats.com/data/anova_sp.csv")

Read function, Section 2.2.1 ,

Representative rows of this data table appear in p. 32 Listing 7.27 . There are 7 participants in the Hi Food quality level, and 7 in the Low Food quality level. For the split-plot design the

identifier for each block of data, here for each Person, re-cycles the same values in the Low Food quality group, even though the reference is for different people. In Listing 7.27 , for example, the identifier p1 appears in both the first row and the 29th row.

> mydata Person Food Supplement Reps 1 p1

Listing 7.27 Data for split-plot factorial design.

Descriptive statistics for the data table can be obtained with the SummaryStats lessR function

function,

SummaryStats , for which the brief form of the output from ss.brief suffices. The summary Section 5.3 , p. 105

178 Compare Multiple Samples

statistics for Food quality appear in Listing 7.28 . The mean repetitions of the bench press for the Hi Food quality participants is 4.36, which is 0.68 repetitions more, on average, than for the Low Food quality participants.

> ss.brief(Reps, by=Food) n miss

Listing 7.28 Marginal means and other summary statistics for the two levels of Food quality.

The marginal means for Supplement are found in Listing 7.29 . The fourth Supplement led to the most repetitions of the bench press with an average of 5.36 bench presses.

> ss.brief(Reps, by=Supplement) n miss

max sup1 14 0 3.21 2.08 1.00 2.00 8.00 sup2 14 0 3.36 1.55 1.00 3.00 6.00 sup3 14 0 4.14 1.96 2.00 4.00 7.00 sup4 14 0 5.36 2.24 3.00 4.50 9.00

Listing 7.29 Marginal means and other summary statistics for the four pre-workout Supplements.

Do these differences observed in the sample also apply to the population? The inferential analysis for this split-plot factorial design proceeds from the R function aov . Food is the between-groups treatment variable and Supplement is the within-groups treatment variable. The interaction.plot function provides the graph of the cell means.

R Input Split-plot factorial design > fit <- aov(Reps ∼ (Food*Supplement) +

Error(Person/Food), data=mydata)

> summary(fit) > with(mydata, interaction.plot(Food, Supplement, Reps)))

The summary table obtained from the aov and summary functions is presented in Listing 7.30 . In this split-plot design the Person variable defines the blocks. From these results, the Food effect on Reps, the number of repetitions of the bench press, is significant.

Food: p -value = 0 . 0401 <α= 0 . 05 , so reject H 0

The more nutritious breakfast does facilitate better performance, which in the sample is 0.68 more bench presses on average.

Compare Multiple Samples 179

Error: Person Df Sum Sq Mean Sq F value Pr(>F) Residuals 6 156.6

Error: Person:Food Df Sum Sq Mean Sq F value Pr(>F) Food

Error: Within Df Sum Sq Mean Sq F value

Pr(>F)

Supplement

Food:Supplement 3 0.91 0.304

Listing 7.30 Split-plot factorial design summary table direct from the R function aov.

The type of Supplement also is significant.

Supplement: p -value = 0 . 0000 <α= 0 . 05 , so reject H 0

As in the analysis of the randomized blocks design, there are differences among the Supplements in terms of facilitating the bench press. There is no detected interaction between Food and Supplement.

Food x Supplement:

p -value = 0 . 802 >α= 0 . 05 , so do not reject H 0

This result means that whatever the effect of Supplement for the Hi nutrition breakfast, the same effect is present for the Low nutrition breakfast. To understand the meaning of this lack of interaction, consider the plot of the cell means in Figure 7.6 .

Supplement

sup4 5.0 sup3 sup1 4.5 sup2

4.0 mean of Reps

Figure 7.6 Cell means of bench press repetitions for two different Food types.

180 Compare Multiple Samples

Analysis of Figure 7.6 reveals approximately parallel lines titled in a downward direction. Regardless of the type of Supplement, there is a shift downward in performance from the Hi to the Low nutrition Food groups. Eating a better breakfast facilitates performance in the gym, but the effects of the different Supplements remain the same regardless of the quality of breakfast.

Worked Problems

?dataEmployee

1 Consider the Employee data set, available from an internal lessR read.

for more information.

> mydata <- Read("Employee", format="lessR") Some of the variables in this data set are Salary, Gender, and Dept for employees at a specific

company. Dept is a categorical variable with five levels: ACCT, ADMN, FINC, MKTG, and SALE.

(a) Are there differences among the average Salary across the five departments? Answer in terms of statistical significance, effect size, and confidence intervals among pairwise differences.

(b) Is this an experiment? Why or why not?

(c) The previous chapter analyzed the difference in average Salary for men and women with an independent-groups t-test. Do so with a one-way ANOVA. Compare the p-values from the t-value from the t-test and the F -value from the ANOVA.

2 Participants in a weight loss program had their weight monitored at the beginning of the program, and at the end of the first, second, and third months. The data are on the web.

> mydata <- Read("http://lessRstats.com/data/WeightLoss4.csv")

(a) Is the weight loss program effective? Answer with both a parametric and non-parametric

procedure, and also effect size. (b) Describe the pattern of weight loss over the three months of the study.

3 Two drugs were evaluated for the relief of anxiety. Each drug was administered in three different dosages. Participants in the study were randomly assigned to one of the six treatment combinations or cells. The data are on the web.

> mydata <- Read("http://lessRstats.com/data/Anxiety.csv")