Effects on Enrollment and Grades of Completed Schooling

A second standard check on RD involves regressions of predetermined character- istics on the indicator value for treatment in our case, children who were offered scholarships and the control function. In general, if the RD design is valid, we would expect the coeffi cient on the treatment indicator in these “placebo” regressions to be small and not signifi cant. Table 1 presents results of these regressions for a number of variables from the household survey that are arguably fi xed—including the gender and age of applicants, education levels of all adults in the household, and various measures of household composition. In the polynomial order one regressions and the LLRs with a bandwidth of six, none of the coeffi cients are signifi cant at the 5 percent level or higher. We conclude that the RD approach we take in this paper is valid, and is likely to provide causal estimates of the medium- term effects of scholarships.

IV. Results

A. Effects on Enrollment and Grades of Completed Schooling

Figure 3 graphs the mean grades attained by scholarship recipients and nonrecipi- ents in 2010, almost fi ve years after both groups completed application forms, and two years after recipients stopped being eligible for scholarships. The fi gure shows a clear jump in grade attainment at the cutoff, equivalent to about half of an additional 2 4 6 8 10 Number of Observations -40 -30 -20 -10 10 20 30 40 Normalized Dropout-risk Score Cutoff point Figure 2 Density Estimates Filmer and Schady 673 Table 1 Applicant Household Characteristics at Endline Polynomial Regression Local Linear Regression Order: 1 Order: 2 Bandwidth = 6 Bandwidth = 3 Bandwidth = 12 Gender –0.051 –0.044 –0.055 –0.032 –0.049 n= 2973 0.027 0.033 0.039 0.060 0.029 Age –0.038 0.054 –0.100 0.048 –0.012 n = 2971 0.073 0.093 0.117 0.160 0.085 Household head education 0.276 –0.087 –0.366 –0.381 –0.003 n = 2706 0.225 0.251 0.317 0.466 0.233 Household spouse education –0.235 –0.425 –0.484 –0.822 –0.338 n = 2013 0.166 0.220 0.260 0.376 0.212 Household highest education 0.137 –0.068 –0.194 –0.175 –0.008 n = 2884 0.216 0.256 0.325 0.442 0.241 Household mean education 0.096 –0.141 –0.375 –0.400 –0.055 n = 2884 0.154 0.184 0.230 0.320 0.169 Number of children 6–11 0.004 –0.027 –0.046 –0.002 –0.026 n = 2973 0.047 0.061 0.072 0.123 0.055 Number of boys 6–11 –0.036 –0.037 –0.033 0.020 –0.035 n = 2973 0.031 0.044 0.053 0.081 0.041 Number of girls 6–11 0.040 0.010 –0.013 –0.021 0.009 n = 2973 0.032 0.042 0.048 0.078 0.038 Number of children 12–18 –0.016 0.037 0.078 0.069 0.032 n = 2973 0.053 0.070 0.081 0.129 0.062 continued The Journal of Human Resources 674 Polynomial Regression Local Linear Regression Order: 1 Order: 2 Bandwidth = 6 Bandwidth = 3 Bandwidth = 12 Number of boys 12–18 –0.064 –0.075 –0.080 –0.077 –0.065 n = 2973 0.054 0.061 0.073 0.097 0.056 Number of girls 12–18 0.048 0.111 0.158 0.147 0.097 n = 2973 0.046 0.058 0.065 0.111 0.054 Number of adults 18–25 –0.012 0.033 0.044 –0.048 0.028 n = 2973 0.054 0.072 0.087 0.146 0.067 Number of adults 25 –0.046 –0.077 –0.081 –0.173 –0.057 n = 2973 0.051 0.067 0.088 0.132 0.062 Notes: Coeffi cients and standard errors from regression of a given characteristic on indicator variable for children with dropout–risk score above the cutoff. Polynomial regres- sion order 1 includes dropout-risk score and its interaction with the indicator for children with scores above the cutoff; polynomial regression order 2 includes dropout-risk score and its square, and interactions with the indicator for children with scores above the cutoff; choice of bandwidth for Local Linear Regressions described in body of paper. Standard errors cluster at level of primary feeder school. Highest mean education calculated by taking the maximum average of education in years per household, only including household members over age 25. p-value 0.01, p-value 0.05, p-value 0.1. Table 1 continued grade of completed schooling. Table 2 reports the results from polynomial regressions and LLRs of grade attainment on the indicator variable for scholarship recipients and the control function. The results suggest a scholarship impact of about 0.6 grades of schooling, relative to a control group mean of 8.35 grades. Table 3 and Figure 4 summarize the results from a number of robustness checks on the results in Table 2. In Figure 4 we present the coeffi cient and 95 percent confi dence interval on the scholarship indicator for LLRs with bandwidths ranging from 1 to 30. The fi gure clearly shows that the results are insensitive to bandwidth choice. As expected, estimates become more precise with larger bandwidths. 7 8 9 10 11 G rade of C o m p le te d School in g -25 -15 -5 5 15 25 Normalized Dropout-risk Score Figure 3 Scholarship Effects on Grades of Completed Schooling Notes: Dashed lines are from polynomial order 1 regression of years of completed schooling on indicator variable for children with dropout- risk score above the cutoff, including secondary school fi xed effects. Circles proportional to number of observations. The Journal of Human Resources 676 Table 2 Scholarship Effects on Years of Completed Schooling Mean: Control Group Polynomial Regression Local Linear Regression Order: 1 Order: 2 Bandwidth = 6 Bandwidth = 3 Bandwidth = 12 Grades of completed schooling 8.35 0.560 0.625 0.598 0.587 0.635 0.101 0.127 0.140 0.210 0.114 N 2,973 2,973 1,869 1,082 2,595 Notes: Coeffi cients and standard errors from regression of grades of completed schooling on indicator variable for children with dropout-risk score above the cutoff. Polyno- mial Regression Order 1 includes dropout-risk score and its interaction with the indicator for children with scores above the cutoff; Polynomial Regression Order 2 includes dropout-risk score and its square, and interactions with the indicator for children with scores above the cutoff; choice of bandwidth for Local Linear Regressions described in body of paper. Standard errors cluster at level of primary feeder school. p-value 0.01, p-value 0.05, p-value 0.1. Table 3 Robustness Checks Interviewed at Midline With Controls Donut 1,1 Donut 2,2 Grades of completed 0.563 0.581 0.522 0.480 schooling 0.101 0.097 0.123 0.149 N 2,944 2,973 2,503 2,084 Notes: Coeffi cients and standard errors from regression of grades of completed schooling on indicator vari- able for children with dropout-risk score above the cutoff, including secondary school fi xed effects. All regressions based on polynomial order 1 specifi cation. Specifi cation in second column includes all controls in Table 1. Standard errors cluster at level of primary feeder school. p-value 0.01, p-value 0.05, p-value 0.1. .5 1 1. 5 Estimated Scholarship Impact on Grades Completed 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 Bandwidth Figure 4 Scholarship Effects on Grades of Completed Schooling, Varying Bandwidths Notes: Coeffi cients and 95 percent confi dence interval are from Local Linear Regression of years of completed schooling on indicator variable for children with dropout- risk score above the cutoff, including secondary school fi xed effects, varying bandwidth. Additional robustness checks based on the polynomial order 1 regressions are in Table 3. One possible concern with our estimates is reporting bias—specifi cally, that scholarship recipients may be more likely to overstate the grades of completed school- ing they have attained than nonrecipients. In earlier work Filmer and Schady 2011 on the short- term effect of scholarships we collected self- reported school enrollment data in a household survey conducted in late 2006, approximately 18 months after children fi lled out the application forms, as well as school attendance data from four unan- nounced school visits we conducted in 2006 and 2007. We showed that the estimated effect of scholarships on enrollment was very similar using both sources of data, sug- gesting that scholarship recipients were no more likely to overstate their school enroll- ment than nonrecipients, at least in the short run. As a robustness test on the results in Table 2 we run regressions using the 2010 household survey data but limit the sample to children who were also interviewed in 2006 and for whom, as discussed above, we had earlier found no evidence of reporting bias. These results are in Column 1 of Table 3. Not surprisingly, given that 99 percent of the children we interviewed in 2010 were also interviewed in 2006, excluding children who were not interviewed in 2006 has no bearing on our results. The following column in Table 3 reports the results from a specifi cation that includes the controls in Table 1. Including these controls does not substantially alter the results. As a fi nal specifi cation check, we report the results from a “donut” esti- mator proposed by Barreca et al. 2011. Barreca et al. point out that the motivating assumption for RD is a comparison of means approaching the cutoff rather than a comparison of means at the cutoff itself . Because any manipulation of eligibility is most likely to occur around the cutoff, Barreca et al. suggest it is useful to check whether RD results are robust to symmetrically discarding observations in the im- mediate vicinity of the cutoff. In Column 3 of Table 3 we report the results from regressions that exclude children whose score places them within one full point of the cutoff; this is equivalent to dropping observations in the two bins that are closest to the cutoff in Figure 2. In Column 4 we exclude those with scores within two points of the cutoff; this is equivalent to dropping observations in the four bins that are closest to the cutoff in Figure 2. These results suggest a scholarship impact of about 0.5 grades of schooling. In sum, the results in Figure 4 and Table 3 show that our main results on grade attainment are robust to a variety of specifi cation checks. In Table 4, we use retrospective questions in the household survey to estimate program effects on school enrollment in grades 7–11. For this purpose, we defi ne the dependent variable in two alternative ways. In the fi rst set of specifi cations, the dependent variable takes on the value of one if a child is enrolled in a given school year, no matter what grade she is enrolled in, and zero otherwise; in the second set of specifi cations, the dependent variable takes on the value of one if a child is enrolled in school in the appropriate grade in a given school year for example, enrolled in grade 7 in 2005–2006, enrolled in grade 8 in 2006–2007, and so on, and zero otherwise. These specifi cations therefore also account for any effect the scholarship may have had on grade repetition in addition to the effect on school enrollment. The fi rst two columns report the mean for the control group. Two things are notable about these results. First, school enrollment declines monotonically and sharply with age. For example, enrollment in the age- appropriate grade declines from 91 percent in grade 7 to 18 percent in grade 11. Second, the differences between the means in the two columns are modest, suggesting that there is relatively little grade repetition in this sample. The remaining columns in Table 4 provide our estimates of scholarship effects. We limit the discussion to the coeffi cients from the polynomial order one and the LLRs with a bandwidth of six, although we note that other results in the table are generally similar. In grade 7, where counterfactual enrollment rates are high, the impact of the scholarship is modest—nine to ten percentage points in our preferred specifi cations. In grades 8 and 9, program effects are substantial, 18 to 20 percentage points. 22 Pro- gram effects in grade 10 are smaller, between seven and eight percentage points, but these represent increases of almost one- third, relative to the control group mean of 23 percent for age- appropriate school enrollment. The results in the last row show even smaller effects on school enrollment in grade 11, and these are not signifi cant at con- ventional levels in our preferred specifi cations. In sum, and consistent with the results on grade attainment discussed earlier, the results in Table 4 show substantial program effects on school enrollment.

B. Effects on Learning Outcomes