The Impact of School Reform on Student Performance Evidence
From the New York Network for School Renewal Project
Amy Ellen Schwartz Leanna Stiefel
Dae Yeop Kim
a b s t r a c t
This paper evaluates the impact of the New York Networks for School Re- newal Project, a whole school reform initiated by the Annenberg Founda-
tion as part of a nationwide reform strategy. It uses data on students in randomly chosen control schools to estimate impacts on student achieve-
ment, using an intent-to-treat design. After controlling for student demo- graphic, mobility, and school characteristics, the authors nd positive im-
pacts for students attending reform schools in the fourth Grade, mixed evidence for fth Grade, and slight to no evidence for sixth Grade. On av-
erage, there is a small positive impact. The paper illustrates how rela- tively inexpensive administrative data can be used to evaluate education
reforms.
I. Introduction
The New York Networks for School Renewal Project NYNSR, ini- tiated with a 25 million, ve-year grant from the Annenberg Foundation, began in
New York City in 1995– 96 with 80 founding schools out of approximately 1000 public schools operating in New York City at that time. NYNSR schools represent
a model of whole school reform that includes a commitment on the part of the schools
Amy Ellen Schwartz is a professor of public policy, Leanna Stiefel is a professor of economics, and Dae Yeop Kim is a doctoral candidate at the Wagner Graduate School, New York University. The au-
thors thank Norm Fruchter, Patrice Iatarola, Dana Lockwood, and Dorothy Siegel for their comments and insight and the Institute for Education and Social Policy at New York University for support and
assistance. The comments of two reviewers were valuable in revising the paper. All responsibility for errors remains with the authors. The data used in this article can be obtained beginning October 2004
through September 2007 from the authors. Email either amy.Schwartznyu.edu
or leanna.stiefelnyu. edu.
[Submitted December 2001; accepted January 2003] ISSN 022-166X; E-ISSN 1548-8004
Ó 2004 by the Board of Regents of the University of Wisconsin System T H E J O U R N A L O F H U M A N R E S O U R C E S X X X I X 2
to work together voluntarily, to enroll small numbers of students in each school, and to adhere to distinct principles of learning. The NYNSR schools span elementary,
middle, and secondary grades in a wide range of grade congurations. In this paper, we evaluate the effect of this reform on the performance of three
cohorts of students —those attending fourth, fth, or sixth Grade in 1995– 96 in founding NYNSR schools. More specically, we use four years of student level and
school level data on student performance, student demographics, school resources, and school characteristics as well as an additional year of pre-reform student test
data to estimate the effect of this school reform on student performance on reading and math tests. The students in NYNSR schools are compared to a control group
of students attending a set of randomly selected schools in 1995– 96. Our cohort studies estimate the impact of attendance in a NYNSR school on the change in a
student’s performance between 1994– 95, the year before the NYNSR initiative be- gan, and subsequent years, through the eighth Grade or their exit from the New York
City public schools. Both short-term one-year and log-term three- or four-year impacts are estimated, and controls are included for various characteristics of stu-
dents and schools.
In addition to providing useful insight into the impact of this reform on student performance, the paper also demonstrates the feasibility and usefulness of using data
obtained from administrative records to evaluate school reforms. The paper is orga- nized as follows. In Section II we discuss whole school reforms in general and the
New York Networks for School Renewal in particular. In Section III we provide a brief review of relevant academic studies that have quantitatively estimated the im-
pact of whole school reforms on student achievement.
1
In Section IV we present a model of student performance for estimating impacts of whole school reforms, and
in Section V we discuss sampling and data. In Section VI we describe the empirical results, and in Section VII we provide concluding comments.
II. Whole School Reform and the New York Networks for School Renewal Project
