where Y
is
is the outcome of interest for child i in school s; and T
s
is an indicator vari- able for whether the school received the reading program. Hence, the estimate of the
coeffi cient β
1
indicates the differences between treatment and control schools. We utilize this model to compare baseline differences in sociodemographic characteristics
and test scores, and to estimate the effect of the reading program on followup test scores and reading habits.
Since the reading program was randomly assigned to schools and therefore indepen- dent of baseline characteristics, inclusion of observable baseline characteristics and
baseline test scores as control variables in Equation 1 improves the precision of the estimated treatment effect. We also run the following specifi cation:
2 Y
is
= α + β
1
T
s
+ β
2
X
is
+ ω
d
+ ε
is
where Y
is
and T
s
are defi ned as in Equation 1, and where X
is
is a vector of baseline stu- dent characteristics including composite baseline reading test score, gender, age, reli-
gion dummies, dialect dummies, and body mass index BMI. Since the randomization was stratifi ed within district, we also include district fi xed effects, ω
d
, in Equation 2. Finally, we test the validity of the experiment by comparing the effect of the treat-
ment on the relative characteristics of the children who attrited from the sample be- tween the baseline survey and the two followup surveys. We run the following differ-
ence in differences model: 3 Y
is
= α + β
1
T
s
+ β
2
Attrit
is
+ β
3
T
s
Attrit
is
+ ε
is
The variables Y
is
and T
s
are defi ned as before, and Attrit
is
is an indicator variable equal to one if student i enrolled in school s was not present in the followup data. The es-
timate of β
2
then provides the average differences between attritors and nonattritors in the control group, and the estimate of
β
3
captures the difference- in- differences between attritors and nonattritors in the treatment and control groups.
Because outcomes may have been correlated within school, failure to correct the standard errors could result in an overestimate of the precision of the treatment effects
Bertrand, Dufl o, and Mullainathan 2004. We therefore cluster the standard errors at the school level the level of randomization in all of the above models.
IV. Internal Validity
Randomly assigning schools to the intervention ensured that assign- ment was orthogonal to student characteristics correlated with the outcomes of interest.
If this holds, then any differences in outcomes between the two groups postinterven- tion can be causally attributed to the intervention. To check that student characteristics
in each group were indeed similar, we run regressions of student characteristics from the baseline survey on treatment assignment, and then we verify that any changes in
the sample due to attrition are also uncorrelated with treatment assignment.
We present the comparison of students at baseline in Table 1. Column 1 contains the average characteristics for the control group. Columns 2 and 3 present the estimated dif-
ferences between the treatment and control groups. The results in Column 2 do not in- clude any controls, while those in Column 3 control for district fi xed effects. Panels A and
B contain standardized reading test scores and demographic characteristics, respectively.
Table 1 Baseline Comparison
Control Mean
Treatment Difference Treatment Difference
No Controls District FE
Dependent Variable 1
2 3
Panel A: Standardized Baseline Reading Test Scores Written test
0.01 0.02
0.03 0.05 0.04
Letter recognition 0.01
0.03 0.03
0.04 0.04 Sound recognition
0.01 –0.10
–0.06 0.07 0.07
Word recognition 0.01
0.02 0.03
0.06 0.06 Oral reading
0.01 0.02
0.04 0.06 0.06
Oral reading questions 0.01
0.04 0.05
0.07 0.05 Average score
0.01 0.01
0.03 0.06
0.06 Panel B: Individual Characteristics
Age 9.37
–0.01 –0.01
0.05 0.05 Female
0.48 –0.01
–0.01 0.01 0.01
Height 128.44
–0.05 –0.05
0.32 0.26 Weight
56.56 0.83
0.57 0.77 0.69
BMI 15.42
0.23 0.15
0.17 0.16 Siblings
3.88 0.07
0.08 0.11 0.09
Catholic 0.74
–0.05 –0.05
0.03 0.02 INC
0.13 0.01
0.01 0.01 0.01
Aglipayan 0.02
0.01 0.01
0.01 0.01
continued
Control Mean
Treatment Difference Treatment Difference
No Controls District FE
Dependent Variable 1
2 3
Born again 0.06
0.02 0.02
0.01 0.01 Protestant
0.02 0.01
0.01 0.01 0.01
Other religion 0.02
0.01 0.01
0.01 0.01
Filipino 0.44
0.01 0.01
0.07 0.04 Iloco
0.19 –0.05
–0.03 0.05 0.03
Kapampangan 0.37
0.04 0.02
0.09 0.03 Pangasinan
0.01 0.01
0.01 0.01
0.01 Other language
0.01 0.01
0.01 0.01
0.01 Observations
2,596 5,510
5,510
Notes: This table presents a comparison of students who took the baseline survey in the control and treatment schools. Column 1 contains the average characteristics of the students in the control schools. Columns 2 and
3 contain estimates of the average difference in characteristics between the control and treatment students, without any controls and with only district fi xed effects. Panel A contains students’ standardized baseline test
scores, and Panel B contains students’ demographic characteristics. Standard errors are clustered by school. indicates statistical signifi cance at the 10 percent level, at the 5 percent level, and at the 1 percent level.
The differences in average characteristics between the control and treatment groups are all practically small and mostly statistically insignifi cant. In Panel A, none of the
differences in test scores are statistically signifi cant. Figure 1 shows a plot of the distri- bution of the standardized overall reading test score for the treatment group solid line
and the control group dashed line. These distributions almost overlap completely, further corroborating the comparability of the research groups. In Panel B, the only
demographic variables with statistically signifi cant differences are those related to religion, but these differences are small in magnitude. For instance, 74 percent of
students in the control group were Catholics compared to 69 percent in the treatment group, yielding a minimal difference of fi ve percentage points. The randomization thus
appears to have successfully created similar treatment and control groups.
Although the baseline comparisons presented in Table 1 and Figure 1 show that the treatment and control groups were similar at baseline, it is possible that nonrandom at-
trition from the two groups between the baseline and followup surveys may have ren- dered the two groups incomparable. Table 2 shows the attrition rates for both groups
and the differences between the two. There are no statistically signifi cant differences
Table 1
continued
between the attrition rates for the control and treatment groups. For both groups, ap- proximately 5 percent of the students who were tested during the baseline survey were
absent during the fi rst followup survey, and 11 percent were absent during the second survey. Comparing the rates across research groups, the rates were the same in the fi rst
followup and differ by only two percentage points in the second ten percentage points in the treatment schools and 12 in the control.
Columns 4–6 provide estimates of the attrition rates between followup surveys. Overall, 86 percent of the students were present at both followup surveys Column 4,
and the difference in the rates between research groups is small. Similarly, 91 percent of students who were present at the fi rst followup were also present at the second, and
of those present at the second, 97 percent were present at the fi rst.
Even though the attrition rates were similar for both groups, the characteristics of the attritors and nonattritors could have still differed. We check this in Table 3 for
the fi rst followup survey. The results for the second followup survey are similar and presented in Table A1 of the online Appendix available at http:jhr.uwpress.org.
Panel A focuses on test scores while Panel B focuses on demographic characteristics. Columns 1 and 2 contain the average characteristic for nonattritors in the control and
treatment groups, respectively, while Column 3 contains the difference between these
Figure 1 Kernel Density Estimates of Baseline Reading Scores
Notes: This fi gure presents kernel density estimates of the baseline total normalized reading score distributions for the treatment and control groups. Distributions estimated using an Epa nechnikov kernel
with a bandwidth of 0.2 standard deviations.
The Journal of Human Resources 620
Table 2 Attrition Rates
Of Baseline Students Number of
Students at Baseline
Fraction at Followup One
Fraction at Followup Two
Fraction at Followup One
and Two Fraction from
Followup One at Followup Two
Fraction from Followup Two
at Followup One 1
2 3
4 5
6 Control schools
2,596 0.95
0.88 0.86
0.90 0.97
Treatment schools 2,914
0.95 0.90
0.87 0.92
0.97 Difference
0.01 0.02
0.02 0.02
0.01 0.01 0.02 0.02 0.02 0.01
Total 5,510
0.95 0.89
0.86 0.91
0.97
Notes: This table shows the fraction of baseline students who took the fi rst and second followup surveys from the control and treatment schools and an estimate of the differ- ence between the two groups. Column 1 contains the number of students who took the baseline survey. Column 2 shows the fraction of baseline students who took the fi rst
followup survey, while Column 3 shows the fraction of baseline students who took the second followup survey. Column 4 contains the fraction of baseline students who took both the fi rst and second followup surveys. Column 5 contains the fraction of baseline students at the fi rst followup survey who took the second followup survey. Column
6 contains the fraction of baseline students at the second followup survey who took the fi rst followup survey. The estimated differences in the third row are estimated using Equation 1. Standard errors are clustered by school and shown in parentheses. indicates statistical signifi cance at the 10 percent level, at the 5 percent level, and at
the 1 percent level.
Abeberese, Kumler
, and Linden
621
Table 3 Attrition Patterns, Followup One
Nonattritors Nonattritors Less Attritors
Control Mean
Treatment Mean
Treatment Difference
Control Difference
Treatment Difference
Difference- in- Difference
Dependent Variable 1
2 3
4 5
6 Panel A: Standardized Baseline Reading Test Scores
Written test 0.01
0.03 0.02
0.10 0.08
–0.01 0.05 0.12
Letter recognition 0.01
0.04 0.04
0.09 0.16
0.07 0.04 0.11
Sound recognition 0.01
–0.09 –0.10
0.22 0.11
–0.11 0.07 0.12
Word recognition 0.01
0.05 0.04
0.25 0.47
0.22 0.06 0.18
Oral reading 0.01
0.04 0.03
0.21 0.43
0.22 0.06 0.21
Oral reading questions 0.01
0.05 0.05
0.06 0.19
0.13 0.07 0.14
Average score 0.01
0.03 0.02
0.22 0.34
0.12 0.06
0.16 continued
The Journal of Human Resources 622
Table 3
continued Nonattritors
Nonattritors Less Attritors Control
Mean Treatment
Mean Treatment
Difference Control
Difference Treatment
Difference Difference- in-
Difference Dependent Variable
1 2
3 4
5 6
Panel B: Individual Characteristics Age
9.32 9.31
–0.01 –0.91
–0.90 0.01
0.04 0.29 Female
0.48 0.47
–0.01 0.07
0.10 0.04
0.01 0.06 Height
128.27 128.27
0.01 –3.34
–2.33 1.00
0.30 1.57 Weight
56.35 57.25
0.90 –4.04
–2.68 1.36
0.79 2.13 BMI
15.41 15.64
0.23 –0.21
–0.07 0.14
0.18 0.32 Siblings
3.84 3.92
0.08 –0.83
–0.65 0.17
0.11 0.25 Catholic
0.74 0.69
–0.05 0.10
0.09 –0.01
0.03 0.07 INC
0.12 0.13
0.01 –0.08
–0.05 0.03
0.01 0.04
Abeberese, Kumler
, and Linden
623 Aglipayan
0.02 0.03
0.01 0.01
0.01 –0.01
0.01 0.02 Born again
0.06 0.08
0.02 0.01
–0.05 –0.04
0.01 0.05 Protestant
0.03 0.04
0.01 0.02
0.01 –0.02
0.01 0.02 Other religion
0.02 0.02
0.01 –0.04
0.01 0.05
0.01 0.02 Filipino
0.44 0.44
0.01 –0.04
–0.01 0.02
0.07 0.07 Iloco
0.19 0.15
–0.04 –0.03
0.06 0.09
0.05 0.05 Kapampangan
0.37 0.40
0.03 0.06
–0.02 –0.08
0.09 0.06 Pangasinan
0.01 0.01
0.01 0.01
–0.01 –0.02
0.01 0.01
Other language 0.01
0.01 0.01
0.01 –0.02
–0.02 0.01
0.02 Observations
2,463 2,765
5,228 2,596
2,914 5,510
Notes: This table presents a comparison of the attrition rates between the treatment and control groups for the fi rst followup survey. Column 1 contains the average charac- teristics of the baseline control students who took the fi rst followup survey while Column 2 contains the average characteristic of the baseline treatment students who took
the fi rst followup survey. Column 3 presents estimates of the average differences in characteristics between the baseline control and treatment students who took the fi rst followup survey. Columns 4 and 5 present the average differences in characteristics between the baseline students who took the fi rst followup survey nonattritors and those
who did not attritors for the control and treatment groups, respectively. Column 6 contains estimates of the differences between the average differences in Columns 4 and 5 using Equation 3. Panel A contains students’ standardized followup one test scores, and Panel B contains students’ demographic characteristics. Standard errors are clustered
by school and shown in parentheses. indicates statistical signifi cance at the 10 percent level, at the 5 percent percent level, and at the 1 percent level.
averages estimated using Equation 1. All of the differences are statistically insignifi - cant with the exception of the proportion of non- attritors who were Catholic. How-
ever, this difference is small in magnitude fi ve percentage points and is identical to the difference found for the entire sample during the baseline survey.
The last three columns of Table 3 show that the differences between the character- istics of the nonattritors and attritors are similar across the two groups, indicating that
there was no selection in the sample due to attrition. Column 4 presents the difference in average characteristic between the nonattritors and the attritors in the control group.
Column 5 presents the same statistic for the treatment group, and Column 6 presents the difference between the two statistics using Equation 3. These differences are mostly
statistically insignifi cant, and all of them are small in magnitude. We therefore conclude that the comparability of the control and treatment groups was sustained throughout the
followup surveys.
V. Results
