Raising the Bar and Equity Effects of St
American Educational Research Association
Raising the Bar and Equity? Effects of State High School Graduation Requirements and
Accountability Policies on Students' Mathematics Course Taking
Author(s): Kathryn S. Schiller and Chandra Muller
Source: Educational Evaluation and Policy Analysis, Vol. 25, No. 3 (Autumn, 2003), pp. 299-318
Published by: American Educational Research Association
Stable URL: http://www.jstor.org/stable/3699497
Accessed: 29/09/2008 14:02
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=aera.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the
scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that
promotes the discovery and use of these resources. For more information about JSTOR, please contact support@jstor.org.
American Educational Research Association is collaborating with JSTOR to digitize, preserve and extend
access to Educational Evaluation and Policy Analysis.
http://www.jstor.org
EducationalEvaluationand Policy Analysis
Fall 2003, Vol. 25, No. 3, pp. 299-318
Raising the Bar and Equity? Effects of State High School
Graduation Requirements and Accountability Policies on
Students' Mathematics Course Taking
Kathryn S. Schiller
State University of New York at Albany
Chandra Muller
University of Texas at Austin
In response to the national push to raise academic performance of all students, most states have
adoptedpolicies designedto raise academicstandards,monitorprogresstowardthose standards,and
hold schools and studentsresponsibleforattainingthem.Giventhe complexnatureof the educational
process, thesepolicies are likely to have mixedeffects on both general levels of attainmentand stratificationbased on race or ethnicityand social class. Using nationallyrepresentativelongitudinaldata
and hierarchicallinear modeling, this article explored the association betweenstudents'mathematics course work and states' high school graduation requirementsand assessment or accountability
policies. Wefound thatstudentsin states withmoregraduationrequirementstendedto enroll in higher
level mathematicscourses asfreshmenandpersist to takemoreadvancedlevel courses. Similartrends
were also found for studentsin states that link test performanceto consequencesfor schools. Extensive testing, however,had little effect on course takingexcept to increase differencesbased on socioeconomicstatus.In contrast,differencesbetweenracial or ethnicgroups tendedto be smallerin states
where testperformancewas linkedto consequencesfor students.
Keywords:accountability,educationalstratification,equity,graduation requirements,mathematics
achievement,opportunitiesto learn, race and ethnicity,social class, sociology of education
THE No Child Left Behind Act of 2001 brought
sweeping changes in the role of the federal government in elementary and secondary schooling
through, among other reforms, increased mandated testing and school accountability. The law
requires states to almost immediately start administering mathematics and reading examinations
based on established state curriculum standards
to all students in grades 3-12. In addition to over-
all progress towardmeeting state standards,the
law also calls for monitoringthe progresswithin
each school of studentswho areeconomicallydisadvantaged,fromracialor ethnicminoritygroups,
have disabilities, or have limited English proficiency. Schools thatfail to makestate-definedadequateprogresstowardmeetingthe statestandards
will be subjectedto increasinglysevere sanctions
over five years culminating with restructuring,
This researchwas supportedby a grantto the first authorfrom the AmericanEducationResearch Foundation,which receives
funds for its "AERA GrantsProgram"from the National Science Foundationand the National Centerfor EducationStatistics
(U.S. Departmentof Education)underNSF GrantRED-9452861. It was also supportedby fundingfrom the NationalInstituteof
Child Healthand HumanDevelopmentundergrantRO1 HD40428-02 (ChandraMuller,PI) andthe NationalScience Foundation
undergrantREC-0126167 (ChandraMuller,PI) to the PopulationResearchCenter,University of Texas at Austin. We thankthe
reviewers for their comments and suggestions. Opinions reflect those of the authorsand do not necessarily reflect those of the
grantingagencies.
299
Schiller and Muller
such as state takeoveror conversionto a charter
school. The goal of this legislation is to not only
raise academicstandardsand, thus, performance
for Americanschool children,butalso to decrease
gaps in achievementbetweensocially advantaged
and disadvantagedgroups.
The provisionsin No ChildLeftBehindwere a
continuationof efforts over the past 40 years by
educationalpolicymakersandpractitioners
to raise
standardsin mathematicsand science. These reforms have included not only raising expectations for students'masteryof these subjects,but
also requiringthat all studentshave exposure to
a core curriculumincorporatingthese standards.
Approximately20 years ago, statepolicy makers
also began implementing examination systems
to hold schools accountable for students' academic progress (McDonnell, 1994). Both reform efforts-raising expectations and increasing externalaccountability-redefinewhata high
school graduate should know and provide incentives for all students to acquire a minimum
level of achievementin orderto earn a diploma.
In response to these reforms, schools have
raised graduationrequirementsand restructured
their academic programs. Between 1980 and
1993, the averagenumberof creditsin core academic subjectsthat schools requiredfor earning
a high school diplomaincreasedby over 1.6 years
(Stevenson & Schiller, 1999). Over two thirds
of this changewas in requirementsfor additional
courses in mathematics and science. Another
dramaticchange was the softening,if not official
elimination, of formal academic tracking systems in favor of a standards-basedcore curriculum in which tracksaremore subjectspecific and
based on timing of course enrollments (Lucas,
1999). During the 1980s, public high schools
increased the size of their academic tracks by
14%to enroll an average of 46% of their sophomore cohortswhile vocationaltrackenrollments
droppedby 12%to an average of less than 19%
of theirsophomorecohorts(Stevenson& Schiller,
1999). This shiftingof studentsinto the academic
track was most dramaticin states requiringtest
score resultsto be widely disseminatedto policy
makers,the media, and parents.
Complexity of the educationalprocess, however,meansthattheseeffortsto improvestudents'
educationalexperiences and academic achievement have had mixed results. High-stakes examinationsfor students,for example, have been
300
relatedto higherratesof droppingout for at-risk
students but do not appear to affect levels of
achievement (Jacob, 2001). Greaterschool accountabilityappearsto increasethe numberof advanced mathematicscreditshigh school students
earn,but does not affect theirprobabilityof earning a diploma(Muller& Schiller,2000). In addition, these stateaccountabilitypolicies also seem
to exacerbatethe attainmentgapbetweenstudents
of low- and high-socioeconomic backgrounds,
suggesting thatpoor studentsmay be negatively
impacted by holding schools responsible for
theiracademicprogress(Muller& Schiller,2000).
Thus, neitherhigh-stakes examinationsfor students nor school accountability are a panacea
for helping all students reach higher academic
standards.
Developing effective policies requiresunderstanding how proposed reforms may influence
studentachievementat differentstages of the educationalprocess, with thoughtfulconsideration
of potentialnegativeeffects. This studyexplores
whetherstudents'mathematicscourseenrollments
as freshmen and in high school overall varied
as a function of states' high school graduation
requirementsand assessment or accountability
policies. Drawing from a nationally representative longitudinal sample of U.S. high school
students in the early 1990s, we used hierarchical linear modeling (HLM) to examine variation across states in both the level of mathematics coursesstudentstendedto takeanddifferences
in course enrollments related to race or ethnicity and social class. We focused on mathematics
because studentsplacementsin this highly structured and sequentialsubjectcreates key turning
points in their opportunitiesto learn (Schneider,
Swanson, & Riegle-Crumb, 1998; Stevenson,
Schiller, & Schneider, 1994). The mathematics
courses studentstake in high school affect their
academic achievement and their admission to
competitive postsecondaryschools and preprofessional programs.
Opportunities for Learning
and State Policies
A core goal of schooling has always been
to promote students' development of skills and
knowledgeimportantfor successas adultsthrough
courses of study providing them with basic opportunitiesfor learning.Since the Cold Warand
A Nation at Risk (National Commission on Ex-
Raising the Bar and Equity?
cellence in Education, 1983), U.S. high schools
have been criticized for failing to producegraduates preparedfor the demandsof highereducation and the workforce.Of particularconcern is
that U.S. high school students continually lag
behind their counterpartsin other industrialized
nations in mathematics and science (Stedman,
1997). The formeris consideredespecially problematic because understandingbasic mathematical principals taught in algebra and geometry
are importantfor students' success in science
(Schmidt et al., 2002). Graduateswho are weak
in these two subjects are consideredunprepared
for entry into medicine, engineering, and other
technology fields. These concerns have focused
policy makers'attentionon whatcoursesstudents
take in high school and whetherthey masterthe
materialto which those courses are supposedto
expose them.
In response to policymakers' mounting concernsaboutbothacademicqualityandeducational
inequity,educationalreformeffortssince the mid1980s have encouraged"de-tracking"by requiring all studentsto complete a common core curriculum (Wells & Oakes, 1996). These efforts
were fueled by sociological research revealing
great variation in the academic experiences of
adolescents, with some exposed to challenging
curriculumin the college preparatorytrackwhile
othersreceived only basic instructionin the general track(Gamoran,1987; Oakes, 1985). While
intended to allow matching of students' talents
andintereststo course content,high school track
assignmentswere often based on non-academic
criteriasuch as social class and ethnicity (Oakes
& Guiton, 1995). Even in schools withoutformal
tracking,students'opportunitiesfor learningare
often constrainedby systemsof prerequisites,especially in highly structuredsubjectslike mathematics, that create sequences of opportunities
for learningthat can span both grade levels and
schools (Stevenson et al., 1994). Where students
areplacedas freshmencreatesa positionaladvantage for gainingaccess to advancedlevel courses,
which are related to greater gains in academic
achievementandentryintopostsecondaryschooling (Schneideret al., 1998).Thus,curricularstructures create defacto tracking in that freshmen
course enrollments determine to a great extent
students'academictrajectoriesin high school. In
this articlewe exploredwhetherstates' efforts to
raise standardsand increaseaccountabilitywere
relatedto the level of mathematicscoursesfreshmen take and how far studentsprogressedin the
subjectduringhigh school.
To what extent policymakers can change students' courseenrollments,and thus achievement,
is questionablebecausemanyindividualfactorsinfluencethe types andnumberof coursesthey take.
Children of college educated parents are more
likely to enroll in algebrain 8th grade,allowing
themto move on to geometryas high schoolfreshmen, comparedto theirclassmateswhose parents
only attendedhigh school (Stevensonet al., 1994;
Useem, 1991). Childrenof moreeducatedparents
not only receive a head startin the high school
mathematicscurriculum,but also tend to persist
in takingcoursesincludingexposureto advanced
algebraandcalculus.While this situationappears
to be changing, girls and minoritystudentshave
in advanced
been traditionallyunder-represented
level mathematicscourses(Oakes, 1985). One of
the centralconcernsof ouranalyseswas to determine whetherstatepolicies were relatedto differences basedon social class andethnicityin freshman mathematicscourse enrollmentsas well as
accumulationof advancedcoursecreditsin these
subjects.
During the early 1980s, regulatory changes
focusedon raisingacademicstandardsby increasing the numberof credits requiredin academic
subjectscomparedto earlierdiplomaholders,in
effect alteringthe definitionof a high school graduate (Chaney, Burgdorf, & Atash, 1997; Clune
& White, 1992; Stevenson & Schiller, 1999).
The logic behind such changes was that requiring students to take more courses in core academic subjects increases their opportunities for
learningkey skills and resultsin higherlevels of
academicachievement.Basic assumptionsbehind
thesepolicies werethatmanyhigh school students
are motivatedto take only the minimumnumber
of requiredcourses, that the additionalcourses
they take will be academicallyrigorous,and that
they areable to do the workrequiredto pass these
courses. Research finds at least partial support
for arguments linking high school graduation
requirementsto increased mathematics course
taking and academic achievement of students,
especially for those who are marginal in their
motivation and skills (Chaneyet al., 1997;Clune
& White, 1992). This studywas designedto compare differences in mathematicstrajectoriesof
similarly able students in states with differing
course requirementsfor high school graduation.
301
Schiller and Muller
Although increased graduationrequirements
appearedto raiseenrollmentin academiccourses,
many policy makersquestionedwhethercourse
titlesaccuratelyreflecttheircontentorthatstudents
may be given passing grades without learning
the material(McDonnell, 1994). These concerns
fueled efforts since the late 1980s to establish
performance standards and increase external
monitoring of students' progress toward those
standards.Initially,mandatingexternalexaminations was a mostly "persuasive"reformstrategy
intendedto provideinformationindicatingwhich
studentsneed remediation,to establishcommon
academicgoals for studentsand teachers,and to
promotecommunitygrassrootsmovements supportingacademicexcellence (McDonnell, 1994).
The assumption underlying these policies was
that regular monitoring of students' academic
progresswould improvetheirpreparationfor advanced level work and increase the demandfor
rigorous course offerings. Critics of these policies, however, raise concerns that standardized
testing creates self-fulling prophecies that limit
the opportunitiesfor learning of academically
and socially disadvantaged students (Wells &
Oakes, 1996). In our studywe examinedwhether
more extensive testing of high school studentsin
academicsubjectswas associatedwith enrollment
in higher level mathematicscourses throughout
high school for all students.
Many states also have establishedformal systems of rewardsand sanctions linked to performance on mandated examinations that are designed to hold studentsand schools accountable
for attainingat least minimalacademicstandards.
Althoughcontroversial,initial reformefforts established high stakes examinations linking test
performanceto consequencesfor individualstudents such as trackplacement,gradepromotion,
and high school graduation(Heubert& Hauser,
1999). Critics expressed concern that such
policies structurallylimit socially and academically disadvantagedstudents' access to opportunities for learningby allocatingthem to remedial courses and encouraging them to dropout
of school (Catterall, 1989; Jacob, 2001). Some
research, however, suggests thatmotivatedlowachievingstudentsmay benefitfroman increased
emphasison academic achievementand support
from teachers(Muller, 1998; Roderick& Engel,
2001; Schiller & Muller, 2000). Because these
examinationsareusuallyheldearlyin high school,
302
the policy is more likely to impact the courses
taken by freshmen and sophomores to prepare
for the tests and might potentially discourage
studentsfromtakingmoreadvancedlevel courses
asjuniorsandseniors.We exploredin theseanalyses whetherstates'high stakesaccountabilitypolicies affected students'tendencies to enroll in or
avoid higher level mathematicscourses at two
key points in theirhigh school careers.
Statepolicies promotinginstitutionalaccountability by linking tangible consequences for
schools to aggregatemeasuresof studentperformance startedbecoming common in the 1990s.
Ratherthan directly regulatinginstructionalactivities, these state policies set academic excellence as the goal while giving schoolsthe freedom
to determinethe best way to help their students
reach the state standards(Elmore, Abelman, &
Fuhrman,1996). One way schools might choose
to raise student performanceis to increase the
numbers enrolled in courses that preparethem
forhigherlevel workin key academicsubjectslike
mathematics.However, some schools might also
marginalizepoor-performing,particularlyminority andpoor, studentsto avoid accountabilityfor
their expected failure on the state assessments
(Schiller & Muller,2000). We exploredwhether
greaterschool accountabilitywas relatedto equity in opportunitiesto learnmathematicsacross
social classes or racialand ethnic groups.
The decentralizednatureof the nation'sschool
systemsmeansthatstatesvarygreatlyin the strategies andpoliciestheyhaveadoptedata giventime.
Although most states raised academic course
requirementsfor a high school diploma during
the 1980s, by 1990 only threestateshad adopted
the NationalCommissionon Excellence in Education's recommendationthat all students take
at least threeyearsof mathematics(Chaneyet al.,
1997). In 1993, states on average required 2.4
years of mathematics (Stevenson & Schiller,
1999). By this time, most states had also implemented some sort of mandatedtesting program,
although how often and in how many subjects
studentswere tested as well as the consequences
for test performancevaried greatly across states
(Schiller & Muller,2000).
In the analysesfor this articlewe examinedthe
impact of greatercourse requirementsfor high
school graduation,more frequentmandatedtesting, andimplementationof sanctionsandrewards
for either studentsor schools linked to test per-
RaisingtheBarandEquity?
formanceon high school students'course taking
in mathematics.Using longitudinaldata from a
nationally representativecohort of 8th graders,
we focused on two key stages of students' academic careers: (a) where they entered the high
school mathematicscurriculumand (b) how far
they progressedthroughthe curriculum.We used
HLM to test the extent to which these policies
aimedat raisinglevels of attainmentandincreasing equity were related to differences based on
social class andrace or ethnicityin opportunities
for learning mathematicsin high school. Controllingfor otheraspectsof students'social backgroundsand middle school mathematicsclasses
and grades, we also examined how the relationshipbetweenfreshmanmathematicscourseplacements and students' persistence in the subject
varied among states with different policies.
Data and Method
Thesample
The analyses in this articlerequiredthe use of
two data sets, one to provide longitudinalinformationon students'social backgroundsand academic experiences,andthe otherto provideinformation on states' assessmentand accountability
policies. Both of the studies we used were conductedin the early 1990s.
The National Education Longitudinal Study
of 1988-92 (NELS:88-92) followed a nationally representativesampleof 8th gradersin 1988
through their high school careers and beyond
(Ingles,Scott,Lindmark,Frankel,& Myers,1992).
The panel used for these analyses consisted of
10,046 public school students who participated
in the firstthree waves of data collection (1988,
1990, and 1992) and for whom high school transcriptswere collected. All 50 states and the Districtof Columbiaarerepresentedin NELS:88-92,
with an average of 196 students and 22 high
schools per state.1Forthese analyses,the sample
was weighted to take into account the complex
sample design and nonresponserates so that the
results would be representativeof those for the
1988 8th-gradecohort.
Informationon states'assessmentandaccountability policies was obtained from the National
LongitudinalStudyof Schools (NLSS). One purpose of NLSS was to examine the impactof state
policies on changes in school practices (Levine
& Stevenson, 1997; Stevenson& Schiller, 1999).
In 1993, state departmentsof education were
asked throughthe National CooperativeEducation Statistics System to answer a lengthy questionnaire concerning their testing and accountability policies. Responses were received from
all 50 states and the Districtof Columbia.
Mathematicscourse enrollments
The measuresof students'mathematicscourse
taking were constructedfrom the NELS:88-92
course-leveltranscriptfile, whichincludesindicators of the topic and when it was takenfor every
course a studenttook duringhigh school. Based
on the standardsequencesof mathematicscourses
most studentstake, courses were classified into
one of the following groups,in hierarchicalorder:
(0) no math,(1) remedialmath,(2) generalmath,
(3) pre-algebra, (4) Algebra I, (5) geometry,
(6) AlgebraII,(7) advancedmath,(8) pre-calculus,
(9) calculus (Schiller & Hunt, 2001). The first
analysis in this articlepredictedthe highest level
mathematicscourse studentstook as freshmen,
indicating where they entered the high school
mathematicscurriculum.The secondanalysispredicted the number of Carnegie units earned in
higherlevel mathematicscourses (geometryand
above), which arecommonlyrequiredfor admission to a competitive college or postsecondary
academic program.Due to a highly structured
sequence of prerequisites,how many Carnegie
units students accumulated in advanced level
mathematicscourses is a good indicatorof how
far they progressed toward calculus. Because
where studentsstartedin the sequencewas likely
to affecthow fartheyprogressed,freshmancourse
placement was also used as a predictor of the
second dependent variable.
Student-levelvariables
This studyfocused on differencesacrossstates
not only in students' freshman mathematics
courses andnumberof advancedcredits,but also
in the variationin these outcomes related to socioeconomic statusandrace or ethnicity.In these
analyses,family socioeconomicstatus(SES) was
a measure of students' financial and social resourcesfrom outsidethe school based on a composite of parents'education,income, andoccupation createdby NCES. Using HLM,we evaluated
whetherthe relationshipbetween SES andmathematics course taking varied across states with
differinggraduationrequirementsor assessment
policies.
303
Schiller and Muller
Insteadof the usual four-groupclassification
of students'race or ethnicity, our HLM models
only included indicators for African American
and Latino/a with the comparison group being
white or Asian American.We chose not to distinguishbetweenwhiteandAsianAmericanstudents
becauseof thelattergroup'ssmallsamplesize and
sparsedistributionin many statesthatresultedin
unreliableHLMcoefficients.Ourresultsconcerning the effects of raceor ethnicityon mathematics
coursetakingandtheirvariationacrossstateswere
not significantlyaffectedby this decision.
To controlfor otherindividualcharacteristics
that might have influenced mathematicscourse
enrollments, we included other indicators of
students'backgroundcharacteristics(genderand
family structure)and prior academic achievement (middle school mathematics grades and
8th-grademathematicscourse enrollments).We
used grades,ratherthantest scores, because they
measurehow well studentsmet the expectations
of their middle school teachers in their classes
and are frequentlyused by high schools to place
studentsin freshmancourses.2We also included
indicatorsfor whetherstudentsattendedan urban
or ruralpublic high school, with suburbanas the
contrastcategory.For a descriptionof these variables, see AppendixA.
Statepolicy measures
Our approachto analyzingthe effects of state
policies was to develop indicatorsof strategies,
or policy levers,thatstatesadoptedto raiseexpectations and increase accountabilityfor students'
academic progress. Efforts to characterizestate
policies have rangedfrom broadgeneralcharacterizationsof the policy environment(Lee, 1998)
to analyses of specific policies such as requiring
studentsto pass an examinationto graduate(Catterall, 1989; Jacob, 2001).3 Analyses using the
former are difficult to interpretbecause distinctions in the purposesof variouspolicies are lost.
The latteroften fail to find significanteffects of
policies unless the analyses focus on the subpopulations most likely subject to the policies.
Our goal was to develop "mid-level"indicators
of statepolicies thatreflectthe variousstrategies
statesused to raise expectationsand accountability as well as the extent to which a type of policy
lever was employed. The measures used in our
analyses were based on statepolicies reportedin
1993, the yearduringwhich most NELS students
graduatedfrom high school.
304
In this studywe includedan indicatorof states'
academic course requirementsfor high school
graduation,the oldest strategy for raising academic standardsby requiring students to take
more courses. In the NLSS questionnaire,states
were asked to report the number of Carnegie
units in various subjects that students were requiredto completeto be eligible for a high school
diploma. The variableused in this analysis was
the total numberof credits requiredin the four
core academicsubjectsof English, social studies,
mathematicsand science. (See AppendixB for a
full descriptionof the statevariables.)Only three
states (Colorado,Massachusetts,andWyoming)
reportedsetting no course requirementsfor high
school graduation,while the remainingstatesrequiredan averageof 9.96 creditsin these subjects
to earna diploma.
Our measure of the extensiveness of states'
testing programsin 1993 was based on their reportsof the gradelevels andmajoracademicsubjects in which mandatedtests were administered
to studentsduringhigh school. Only seven states
reported no mandated testing of high school
studentsin the major subjects of English, social
studies/history,mathematics, and science. The
remainingstatesgave on averagefourteststo high
school students,althoughtwo states (Minnesota
andVirginia)reportedtesting studentsin all four
subjectseveryyear.The extensivenessof a state's
testing programis an indicatorof whetherexternal examinations were used on a regular basis
to monitorstudents'progressthroughthe established curriculum,usually with the intention of
raising overall levels of achievement.
Although most states tested high school students, they varied in the extent to which performance on those tests carriedmeaningfulconsequences for studentsor schools. Ourmeasureof
consequences for studentsbased on test performance was the sum of states' reportsof whether
test scoreswererecommendedor requiredforpurposes such as placementin remedialor advanced
placementprograms,promotionto the next grade,
or awardof a high school diploma. Almost two
thirdsof the states had guidelines or mandatory
policies specifyinghow test scoresshouldbe used
to determinesome aspect of students' academic
program or success. Those states with such
policies linked test scores to an average of three
or four consequences for students. Fewer states
linked students'performanceon mandatedtests
Raising the Bar and Equity?
to rewardsandsanctionsfor schools in 1993. The
survey asked abouteight types of consequences,
such as financialrewardsfor meeting standards
or sanctionslike loss of accreditationfor failure
to do so. Over two thirds of the states reported
thatthey eitherdid not set performancestandards
or did not provide incentives for meeting those
standards.The remainingthirdof the stateslinked
aggregatetest scoresto an averageof threeor four
consequencesfor schools.
In preliminaryanalyses, the four measuresof
states' policies appearedto reflectdistinctstrategies for increasingstandardsand accountability.
The four measures of state policies were only
moderately related to each other with correlations all less than .36. The strongestcorrelation
reflected that the number of consequences for
schools and for students linked to test performance was relatedto stateshaving a testing program.However, the extensiveness of testing and
how results were used to determinesanctionsor
rewardstendedto be unrelated.
Analysis technique
The questionsof whetherstatetestingpolicies
were relatedto students'mathematicscourses in
high school requireda multilevel analytic strategy. We were concernednot only with variation
in students' mathematics course taking across
states with differingpolicies (directeffects), but
also with whether the associations of students'
outcomes with their social backgroundsvaried
across states (interaction effects). A common
techniquefor analyzinghierarchicaldata(in this
case, students nested within states) and crosslevel effects is HLM, which allows simultaneous considerationof factors from two levels of
analysis (Bryk & Raudenbush, 1992; Raudenbush & Bryk, 1986).4
The same student and state policy variables
were used for analyses of students' freshman
mathematicscourse level and the numberof advancedmathematicscreditsearned,except freshman course level was also used to predict the
lateroutcome.The student-levelmodel is shown
in Equation1, where ij was the value for a given
studentin a given stateandBkjwas the coefficient
for students'SES, race and ethnicity,or the control variablesin each state. The effects for some
of the student-levelfactors, such as race or ethnicity, were expressedby severalcoefficients,for
example B2j for Latino/a and B3j for African-
American.The termeijwas a measureof the random error,which included unmeasuredsources
of variationin a particularstudent'soutcome. In
our analyses, all the student-levelvariableswere
centered aroundtheir grandmeans for the sample, which allowed the intercept(Boj)to be interpretedas the meanoutcomefor each stateadjusted
for the characteristics of students in that state
(Bryk, Raudenbush,& Congdon, 1996; Willms
& Raudenbush,1989).
Y, = P,j + By (SESi,)
+ B2j-3j(Race/Ethnicityij)
+ B4j-0j (Controls) + ei
(1)
Preliminaryanalysesindicatedthat,in oursample, the associations between the student-level
controlvariablesandmathematicscourseseither
did not vary significantly across states or those
variationswere not related to state testing policies. Either situation meant that assuming the
associations were constant across states did not
substantiallyaffect the results for SES and race
or ethnicity. Thus, for the analyses presented
here,the coefficientsfor the student-levelcontrol
variables were set to be "fixed effects" and our
statistical model assumed that the relationships
between these studentcharacteristicsandmathematics course enrollmentswere the same for all
states (Bryk & Raudenbush,1992).
The state-levelanalyses,in essence, examined
the extent to which variationin the coefficients
for the intercept,SES, andrace or ethnicitywere
related to states' graduationand accountability
policies. Equation 2 shows the general model
used for estimatingthe effects of these statepolicies.5 Each of the policy variableswas centered
aroundits grandmean, which meantthatYk0was
the averageeffect of variablek across states and
the other coefficients were adjustmentsto those
coefficients, or interactioneffects, for states that
differed in their testing policies. The effect of a
student-level variable was increased when the
coefficient for a state policy variablewas in the
same direction (plus or minus) as the intercept
for the student-levelvariableand reducedwhen
the two coefficients were in the opposite direction. The termukjwas the errorterm for estimation of the student-levelcoefficientfor each state.
Bkj = YkO + Ykl-k4(State
Policiesj) +ukj
(2)
305
Schiller and Muller
The combined HLM model is shown in
Equation 3.
Yi = [Too+ Yoi-o4(State Policiesj)+
oj]
+[710 + Y1-04(State Policiesj)+
+[2-30
+
22-34(State Policiesj)
lj]*SESi
+U23j]
* Race/Ethnicityij
+
4-100* Control Variablesi
+eij
(3)
Results
States vary in their approachesto raisinghigh
school students' academic attainmentand promoting equalityof opportunitiesfor learning,but
the extentto whichthesepolicies impactstudents'
educationalcareersis uncertain.The purposeof
our study was to examine variationacross states
adoptingdifferentstrategiesfor raisingstandards
and establishingaccountabilityin two criticalaspectsof students'mathematicscourseenrollments:
wherethey startedas freshmen,andthe amountof
advanced-levelcourseworkcompletedby graduation. The goal was to determinewhetherthese
statepolicies were relatedto students'mathematics course placementsas freshmenand theirpersistence in advancedmathematicsas well as differencesbased on SES and race or ethnicity.
Freshman Mathematics Course Placements
Theresultsfor students'freshmanmathematics
courseenrollmentsare shownin Table 1. The top
panel of the table containsthe coefficientsfor the
interceptand independentvariablesmodeled on
the statelevel. The firstcolumn shows the Level2 intercept,or averageeffect, for the student-level
variables of interest. The other four columns
show the coefficients for state policy variables.
The lower panel containsthe coefficients for the
student-levelcontrols,which were assumedto be
constantacross states.
All of the student-levelcontrolvariablesexcept
urbanicitywere significantpredictorsof students'
mathematicscourseplacementsas freshmen.Students tendedto enroll in a higher level course if
they were female, lived with bothnaturalparents,
had highermathematicsgradesin middle school,
and enrolledin Algebraas an 8th grader.6Eighth
graderswho took remedial mathematicstended
to be placed in lower level courses as freshmen,
306
even takinginto accounttheirsocial backgrounds
andmiddleschool mathematicsgrades.Freshman
courseenrollmentsappearedto have been similar
acrossurban,suburban,andrurallocations.
The first row of Table 1 indicates that states'
graduationand accountabilitypolicies were related to differences in where freshmenenter the
high school mathematics sequence. On average, freshmen tended to enroll in pre-algebra
(coded as 3). In states requiringmore academic
coursecreditsfor graduation,freshmentendedto
take slightly higher level mathematicscourses.
Although statisticallysignificant,this effect was
small at less than 7% of a course level per standarddeviation change in the numberof courses
required(.077 = .024 * 3.203). This difference,
however, was only slightly smaller than those
relatedto genderor family structure.Extensiveness of testing was also significantly related to
freshman course enrollments, with students in
states with more extensive testing tendingto enroll in slightlylower level freshmanmathematics
courses (-.059 = -.017 * 3.461). Neither of the
state accountabilitypolicy variableswere significantly relatedto freshmancourse enrollments.
Extensivetestingwas also significantlyrelated
to a somewhat strongereffect of socioeconomic
status on freshmanmathematicscourse level. A
standarddeviationincreasein the numberof tests
was relatedto almosta 20%increasein the effect
of SES [.197 = (.018 * 3.461)/.314]. These results indicatethe gaps between poor andrich students were larger in states that test high school
students more frequently and in more subjects.
The strongereffect of SES in states with more
extensive testing was consistentwith analysesof
otheracademicoutcomes such as earninga high
school diploma(Muller& Schiller, 2000).
Our results identified no overall differences
based on race or ethnicity after controlling for
prioracademicperformanceand SES. However,
statepolicieswererelatedto significantdifferences
in freshman mathematics course enrollments
between African American and white students.
The tendency for African American studentsto
enroll in somewhat lower level courses comparedto similarwhites was strongerin statesrequiring more academic courses for graduation
or linking test performanceto consequences for
students. The latter policy strategy more than
doubled the effect of being African American
for each additionalconsequence.Conversely,the
TABLE 1
Effectsof State Policies on the Level of FreshmanMathematicsCourses
State Policy
Student-levelVariable
Average Effect
Intercept
Socioeconomic status
3.507***
.314***
Race/ethnicity
Latino/a
AfricanAmerican
-.030
-.074
Student-levelControl
Male
Living with both parents
Middle school mathgrades
8th-grademathclass
Remedial
Algebra/advanced
Urbanicity
Urban
Rural
* =p < .05, ** =p
Raising the Bar and Equity? Effects of State High School Graduation Requirements and
Accountability Policies on Students' Mathematics Course Taking
Author(s): Kathryn S. Schiller and Chandra Muller
Source: Educational Evaluation and Policy Analysis, Vol. 25, No. 3 (Autumn, 2003), pp. 299-318
Published by: American Educational Research Association
Stable URL: http://www.jstor.org/stable/3699497
Accessed: 29/09/2008 14:02
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=aera.
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.
JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the
scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that
promotes the discovery and use of these resources. For more information about JSTOR, please contact support@jstor.org.
American Educational Research Association is collaborating with JSTOR to digitize, preserve and extend
access to Educational Evaluation and Policy Analysis.
http://www.jstor.org
EducationalEvaluationand Policy Analysis
Fall 2003, Vol. 25, No. 3, pp. 299-318
Raising the Bar and Equity? Effects of State High School
Graduation Requirements and Accountability Policies on
Students' Mathematics Course Taking
Kathryn S. Schiller
State University of New York at Albany
Chandra Muller
University of Texas at Austin
In response to the national push to raise academic performance of all students, most states have
adoptedpolicies designedto raise academicstandards,monitorprogresstowardthose standards,and
hold schools and studentsresponsibleforattainingthem.Giventhe complexnatureof the educational
process, thesepolicies are likely to have mixedeffects on both general levels of attainmentand stratificationbased on race or ethnicityand social class. Using nationallyrepresentativelongitudinaldata
and hierarchicallinear modeling, this article explored the association betweenstudents'mathematics course work and states' high school graduation requirementsand assessment or accountability
policies. Wefound thatstudentsin states withmoregraduationrequirementstendedto enroll in higher
level mathematicscourses asfreshmenandpersist to takemoreadvancedlevel courses. Similartrends
were also found for studentsin states that link test performanceto consequencesfor schools. Extensive testing, however,had little effect on course takingexcept to increase differencesbased on socioeconomicstatus.In contrast,differencesbetweenracial or ethnicgroups tendedto be smallerin states
where testperformancewas linkedto consequencesfor students.
Keywords:accountability,educationalstratification,equity,graduation requirements,mathematics
achievement,opportunitiesto learn, race and ethnicity,social class, sociology of education
THE No Child Left Behind Act of 2001 brought
sweeping changes in the role of the federal government in elementary and secondary schooling
through, among other reforms, increased mandated testing and school accountability. The law
requires states to almost immediately start administering mathematics and reading examinations
based on established state curriculum standards
to all students in grades 3-12. In addition to over-
all progress towardmeeting state standards,the
law also calls for monitoringthe progresswithin
each school of studentswho areeconomicallydisadvantaged,fromracialor ethnicminoritygroups,
have disabilities, or have limited English proficiency. Schools thatfail to makestate-definedadequateprogresstowardmeetingthe statestandards
will be subjectedto increasinglysevere sanctions
over five years culminating with restructuring,
This researchwas supportedby a grantto the first authorfrom the AmericanEducationResearch Foundation,which receives
funds for its "AERA GrantsProgram"from the National Science Foundationand the National Centerfor EducationStatistics
(U.S. Departmentof Education)underNSF GrantRED-9452861. It was also supportedby fundingfrom the NationalInstituteof
Child Healthand HumanDevelopmentundergrantRO1 HD40428-02 (ChandraMuller,PI) andthe NationalScience Foundation
undergrantREC-0126167 (ChandraMuller,PI) to the PopulationResearchCenter,University of Texas at Austin. We thankthe
reviewers for their comments and suggestions. Opinions reflect those of the authorsand do not necessarily reflect those of the
grantingagencies.
299
Schiller and Muller
such as state takeoveror conversionto a charter
school. The goal of this legislation is to not only
raise academicstandardsand, thus, performance
for Americanschool children,butalso to decrease
gaps in achievementbetweensocially advantaged
and disadvantagedgroups.
The provisionsin No ChildLeftBehindwere a
continuationof efforts over the past 40 years by
educationalpolicymakersandpractitioners
to raise
standardsin mathematicsand science. These reforms have included not only raising expectations for students'masteryof these subjects,but
also requiringthat all studentshave exposure to
a core curriculumincorporatingthese standards.
Approximately20 years ago, statepolicy makers
also began implementing examination systems
to hold schools accountable for students' academic progress (McDonnell, 1994). Both reform efforts-raising expectations and increasing externalaccountability-redefinewhata high
school graduate should know and provide incentives for all students to acquire a minimum
level of achievementin orderto earn a diploma.
In response to these reforms, schools have
raised graduationrequirementsand restructured
their academic programs. Between 1980 and
1993, the averagenumberof creditsin core academic subjectsthat schools requiredfor earning
a high school diplomaincreasedby over 1.6 years
(Stevenson & Schiller, 1999). Over two thirds
of this changewas in requirementsfor additional
courses in mathematics and science. Another
dramaticchange was the softening,if not official
elimination, of formal academic tracking systems in favor of a standards-basedcore curriculum in which tracksaremore subjectspecific and
based on timing of course enrollments (Lucas,
1999). During the 1980s, public high schools
increased the size of their academic tracks by
14%to enroll an average of 46% of their sophomore cohortswhile vocationaltrackenrollments
droppedby 12%to an average of less than 19%
of theirsophomorecohorts(Stevenson& Schiller,
1999). This shiftingof studentsinto the academic
track was most dramaticin states requiringtest
score resultsto be widely disseminatedto policy
makers,the media, and parents.
Complexity of the educationalprocess, however,meansthattheseeffortsto improvestudents'
educationalexperiences and academic achievement have had mixed results. High-stakes examinationsfor students,for example, have been
300
relatedto higherratesof droppingout for at-risk
students but do not appear to affect levels of
achievement (Jacob, 2001). Greaterschool accountabilityappearsto increasethe numberof advanced mathematicscreditshigh school students
earn,but does not affect theirprobabilityof earning a diploma(Muller& Schiller,2000). In addition, these stateaccountabilitypolicies also seem
to exacerbatethe attainmentgapbetweenstudents
of low- and high-socioeconomic backgrounds,
suggesting thatpoor studentsmay be negatively
impacted by holding schools responsible for
theiracademicprogress(Muller& Schiller,2000).
Thus, neitherhigh-stakes examinationsfor students nor school accountability are a panacea
for helping all students reach higher academic
standards.
Developing effective policies requiresunderstanding how proposed reforms may influence
studentachievementat differentstages of the educationalprocess, with thoughtfulconsideration
of potentialnegativeeffects. This studyexplores
whetherstudents'mathematicscourseenrollments
as freshmen and in high school overall varied
as a function of states' high school graduation
requirementsand assessment or accountability
policies. Drawing from a nationally representative longitudinal sample of U.S. high school
students in the early 1990s, we used hierarchical linear modeling (HLM) to examine variation across states in both the level of mathematics coursesstudentstendedto takeanddifferences
in course enrollments related to race or ethnicity and social class. We focused on mathematics
because studentsplacementsin this highly structured and sequentialsubjectcreates key turning
points in their opportunitiesto learn (Schneider,
Swanson, & Riegle-Crumb, 1998; Stevenson,
Schiller, & Schneider, 1994). The mathematics
courses studentstake in high school affect their
academic achievement and their admission to
competitive postsecondaryschools and preprofessional programs.
Opportunities for Learning
and State Policies
A core goal of schooling has always been
to promote students' development of skills and
knowledgeimportantfor successas adultsthrough
courses of study providing them with basic opportunitiesfor learning.Since the Cold Warand
A Nation at Risk (National Commission on Ex-
Raising the Bar and Equity?
cellence in Education, 1983), U.S. high schools
have been criticized for failing to producegraduates preparedfor the demandsof highereducation and the workforce.Of particularconcern is
that U.S. high school students continually lag
behind their counterpartsin other industrialized
nations in mathematics and science (Stedman,
1997). The formeris consideredespecially problematic because understandingbasic mathematical principals taught in algebra and geometry
are importantfor students' success in science
(Schmidt et al., 2002). Graduateswho are weak
in these two subjects are consideredunprepared
for entry into medicine, engineering, and other
technology fields. These concerns have focused
policy makers'attentionon whatcoursesstudents
take in high school and whetherthey masterthe
materialto which those courses are supposedto
expose them.
In response to policymakers' mounting concernsaboutbothacademicqualityandeducational
inequity,educationalreformeffortssince the mid1980s have encouraged"de-tracking"by requiring all studentsto complete a common core curriculum (Wells & Oakes, 1996). These efforts
were fueled by sociological research revealing
great variation in the academic experiences of
adolescents, with some exposed to challenging
curriculumin the college preparatorytrackwhile
othersreceived only basic instructionin the general track(Gamoran,1987; Oakes, 1985). While
intended to allow matching of students' talents
andintereststo course content,high school track
assignmentswere often based on non-academic
criteriasuch as social class and ethnicity (Oakes
& Guiton, 1995). Even in schools withoutformal
tracking,students'opportunitiesfor learningare
often constrainedby systemsof prerequisites,especially in highly structuredsubjectslike mathematics, that create sequences of opportunities
for learningthat can span both grade levels and
schools (Stevenson et al., 1994). Where students
areplacedas freshmencreatesa positionaladvantage for gainingaccess to advancedlevel courses,
which are related to greater gains in academic
achievementandentryintopostsecondaryschooling (Schneideret al., 1998).Thus,curricularstructures create defacto tracking in that freshmen
course enrollments determine to a great extent
students'academictrajectoriesin high school. In
this articlewe exploredwhetherstates' efforts to
raise standardsand increaseaccountabilitywere
relatedto the level of mathematicscoursesfreshmen take and how far studentsprogressedin the
subjectduringhigh school.
To what extent policymakers can change students' courseenrollments,and thus achievement,
is questionablebecausemanyindividualfactorsinfluencethe types andnumberof coursesthey take.
Children of college educated parents are more
likely to enroll in algebrain 8th grade,allowing
themto move on to geometryas high schoolfreshmen, comparedto theirclassmateswhose parents
only attendedhigh school (Stevensonet al., 1994;
Useem, 1991). Childrenof moreeducatedparents
not only receive a head startin the high school
mathematicscurriculum,but also tend to persist
in takingcoursesincludingexposureto advanced
algebraandcalculus.While this situationappears
to be changing, girls and minoritystudentshave
in advanced
been traditionallyunder-represented
level mathematicscourses(Oakes, 1985). One of
the centralconcernsof ouranalyseswas to determine whetherstatepolicies were relatedto differences basedon social class andethnicityin freshman mathematicscourse enrollmentsas well as
accumulationof advancedcoursecreditsin these
subjects.
During the early 1980s, regulatory changes
focusedon raisingacademicstandardsby increasing the numberof credits requiredin academic
subjectscomparedto earlierdiplomaholders,in
effect alteringthe definitionof a high school graduate (Chaney, Burgdorf, & Atash, 1997; Clune
& White, 1992; Stevenson & Schiller, 1999).
The logic behind such changes was that requiring students to take more courses in core academic subjects increases their opportunities for
learningkey skills and resultsin higherlevels of
academicachievement.Basic assumptionsbehind
thesepolicies werethatmanyhigh school students
are motivatedto take only the minimumnumber
of requiredcourses, that the additionalcourses
they take will be academicallyrigorous,and that
they areable to do the workrequiredto pass these
courses. Research finds at least partial support
for arguments linking high school graduation
requirementsto increased mathematics course
taking and academic achievement of students,
especially for those who are marginal in their
motivation and skills (Chaneyet al., 1997;Clune
& White, 1992). This studywas designedto compare differences in mathematicstrajectoriesof
similarly able students in states with differing
course requirementsfor high school graduation.
301
Schiller and Muller
Although increased graduationrequirements
appearedto raiseenrollmentin academiccourses,
many policy makersquestionedwhethercourse
titlesaccuratelyreflecttheircontentorthatstudents
may be given passing grades without learning
the material(McDonnell, 1994). These concerns
fueled efforts since the late 1980s to establish
performance standards and increase external
monitoring of students' progress toward those
standards.Initially,mandatingexternalexaminations was a mostly "persuasive"reformstrategy
intendedto provideinformationindicatingwhich
studentsneed remediation,to establishcommon
academicgoals for studentsand teachers,and to
promotecommunitygrassrootsmovements supportingacademicexcellence (McDonnell, 1994).
The assumption underlying these policies was
that regular monitoring of students' academic
progresswould improvetheirpreparationfor advanced level work and increase the demandfor
rigorous course offerings. Critics of these policies, however, raise concerns that standardized
testing creates self-fulling prophecies that limit
the opportunitiesfor learning of academically
and socially disadvantaged students (Wells &
Oakes, 1996). In our studywe examinedwhether
more extensive testing of high school studentsin
academicsubjectswas associatedwith enrollment
in higher level mathematicscourses throughout
high school for all students.
Many states also have establishedformal systems of rewardsand sanctions linked to performance on mandated examinations that are designed to hold studentsand schools accountable
for attainingat least minimalacademicstandards.
Althoughcontroversial,initial reformefforts established high stakes examinations linking test
performanceto consequencesfor individualstudents such as trackplacement,gradepromotion,
and high school graduation(Heubert& Hauser,
1999). Critics expressed concern that such
policies structurallylimit socially and academically disadvantagedstudents' access to opportunities for learningby allocatingthem to remedial courses and encouraging them to dropout
of school (Catterall, 1989; Jacob, 2001). Some
research, however, suggests thatmotivatedlowachievingstudentsmay benefitfroman increased
emphasison academic achievementand support
from teachers(Muller, 1998; Roderick& Engel,
2001; Schiller & Muller, 2000). Because these
examinationsareusuallyheldearlyin high school,
302
the policy is more likely to impact the courses
taken by freshmen and sophomores to prepare
for the tests and might potentially discourage
studentsfromtakingmoreadvancedlevel courses
asjuniorsandseniors.We exploredin theseanalyses whetherstates'high stakesaccountabilitypolicies affected students'tendencies to enroll in or
avoid higher level mathematicscourses at two
key points in theirhigh school careers.
Statepolicies promotinginstitutionalaccountability by linking tangible consequences for
schools to aggregatemeasuresof studentperformance startedbecoming common in the 1990s.
Ratherthan directly regulatinginstructionalactivities, these state policies set academic excellence as the goal while giving schoolsthe freedom
to determinethe best way to help their students
reach the state standards(Elmore, Abelman, &
Fuhrman,1996). One way schools might choose
to raise student performanceis to increase the
numbers enrolled in courses that preparethem
forhigherlevel workin key academicsubjectslike
mathematics.However, some schools might also
marginalizepoor-performing,particularlyminority andpoor, studentsto avoid accountabilityfor
their expected failure on the state assessments
(Schiller & Muller,2000). We exploredwhether
greaterschool accountabilitywas relatedto equity in opportunitiesto learnmathematicsacross
social classes or racialand ethnic groups.
The decentralizednatureof the nation'sschool
systemsmeansthatstatesvarygreatlyin the strategies andpoliciestheyhaveadoptedata giventime.
Although most states raised academic course
requirementsfor a high school diploma during
the 1980s, by 1990 only threestateshad adopted
the NationalCommissionon Excellence in Education's recommendationthat all students take
at least threeyearsof mathematics(Chaneyet al.,
1997). In 1993, states on average required 2.4
years of mathematics (Stevenson & Schiller,
1999). By this time, most states had also implemented some sort of mandatedtesting program,
although how often and in how many subjects
studentswere tested as well as the consequences
for test performancevaried greatly across states
(Schiller & Muller,2000).
In the analysesfor this articlewe examinedthe
impact of greatercourse requirementsfor high
school graduation,more frequentmandatedtesting, andimplementationof sanctionsandrewards
for either studentsor schools linked to test per-
RaisingtheBarandEquity?
formanceon high school students'course taking
in mathematics.Using longitudinaldata from a
nationally representativecohort of 8th graders,
we focused on two key stages of students' academic careers: (a) where they entered the high
school mathematicscurriculumand (b) how far
they progressedthroughthe curriculum.We used
HLM to test the extent to which these policies
aimedat raisinglevels of attainmentandincreasing equity were related to differences based on
social class andrace or ethnicityin opportunities
for learning mathematicsin high school. Controllingfor otheraspectsof students'social backgroundsand middle school mathematicsclasses
and grades, we also examined how the relationshipbetweenfreshmanmathematicscourseplacements and students' persistence in the subject
varied among states with different policies.
Data and Method
Thesample
The analyses in this articlerequiredthe use of
two data sets, one to provide longitudinalinformationon students'social backgroundsand academic experiences,andthe otherto provideinformation on states' assessmentand accountability
policies. Both of the studies we used were conductedin the early 1990s.
The National Education Longitudinal Study
of 1988-92 (NELS:88-92) followed a nationally representativesampleof 8th gradersin 1988
through their high school careers and beyond
(Ingles,Scott,Lindmark,Frankel,& Myers,1992).
The panel used for these analyses consisted of
10,046 public school students who participated
in the firstthree waves of data collection (1988,
1990, and 1992) and for whom high school transcriptswere collected. All 50 states and the Districtof Columbiaarerepresentedin NELS:88-92,
with an average of 196 students and 22 high
schools per state.1Forthese analyses,the sample
was weighted to take into account the complex
sample design and nonresponserates so that the
results would be representativeof those for the
1988 8th-gradecohort.
Informationon states'assessmentandaccountability policies was obtained from the National
LongitudinalStudyof Schools (NLSS). One purpose of NLSS was to examine the impactof state
policies on changes in school practices (Levine
& Stevenson, 1997; Stevenson& Schiller, 1999).
In 1993, state departmentsof education were
asked throughthe National CooperativeEducation Statistics System to answer a lengthy questionnaire concerning their testing and accountability policies. Responses were received from
all 50 states and the Districtof Columbia.
Mathematicscourse enrollments
The measuresof students'mathematicscourse
taking were constructedfrom the NELS:88-92
course-leveltranscriptfile, whichincludesindicators of the topic and when it was takenfor every
course a studenttook duringhigh school. Based
on the standardsequencesof mathematicscourses
most studentstake, courses were classified into
one of the following groups,in hierarchicalorder:
(0) no math,(1) remedialmath,(2) generalmath,
(3) pre-algebra, (4) Algebra I, (5) geometry,
(6) AlgebraII,(7) advancedmath,(8) pre-calculus,
(9) calculus (Schiller & Hunt, 2001). The first
analysis in this articlepredictedthe highest level
mathematicscourse studentstook as freshmen,
indicating where they entered the high school
mathematicscurriculum.The secondanalysispredicted the number of Carnegie units earned in
higherlevel mathematicscourses (geometryand
above), which arecommonlyrequiredfor admission to a competitive college or postsecondary
academic program.Due to a highly structured
sequence of prerequisites,how many Carnegie
units students accumulated in advanced level
mathematicscourses is a good indicatorof how
far they progressed toward calculus. Because
where studentsstartedin the sequencewas likely
to affecthow fartheyprogressed,freshmancourse
placement was also used as a predictor of the
second dependent variable.
Student-levelvariables
This studyfocused on differencesacrossstates
not only in students' freshman mathematics
courses andnumberof advancedcredits,but also
in the variationin these outcomes related to socioeconomic statusandrace or ethnicity.In these
analyses,family socioeconomicstatus(SES) was
a measure of students' financial and social resourcesfrom outsidethe school based on a composite of parents'education,income, andoccupation createdby NCES. Using HLM,we evaluated
whetherthe relationshipbetween SES andmathematics course taking varied across states with
differinggraduationrequirementsor assessment
policies.
303
Schiller and Muller
Insteadof the usual four-groupclassification
of students'race or ethnicity, our HLM models
only included indicators for African American
and Latino/a with the comparison group being
white or Asian American.We chose not to distinguishbetweenwhiteandAsianAmericanstudents
becauseof thelattergroup'ssmallsamplesize and
sparsedistributionin many statesthatresultedin
unreliableHLMcoefficients.Ourresultsconcerning the effects of raceor ethnicityon mathematics
coursetakingandtheirvariationacrossstateswere
not significantlyaffectedby this decision.
To controlfor otherindividualcharacteristics
that might have influenced mathematicscourse
enrollments, we included other indicators of
students'backgroundcharacteristics(genderand
family structure)and prior academic achievement (middle school mathematics grades and
8th-grademathematicscourse enrollments).We
used grades,ratherthantest scores, because they
measurehow well studentsmet the expectations
of their middle school teachers in their classes
and are frequentlyused by high schools to place
studentsin freshmancourses.2We also included
indicatorsfor whetherstudentsattendedan urban
or ruralpublic high school, with suburbanas the
contrastcategory.For a descriptionof these variables, see AppendixA.
Statepolicy measures
Our approachto analyzingthe effects of state
policies was to develop indicatorsof strategies,
or policy levers,thatstatesadoptedto raiseexpectations and increase accountabilityfor students'
academic progress. Efforts to characterizestate
policies have rangedfrom broadgeneralcharacterizationsof the policy environment(Lee, 1998)
to analyses of specific policies such as requiring
studentsto pass an examinationto graduate(Catterall, 1989; Jacob, 2001).3 Analyses using the
former are difficult to interpretbecause distinctions in the purposesof variouspolicies are lost.
The latteroften fail to find significanteffects of
policies unless the analyses focus on the subpopulations most likely subject to the policies.
Our goal was to develop "mid-level"indicators
of statepolicies thatreflectthe variousstrategies
statesused to raise expectationsand accountability as well as the extent to which a type of policy
lever was employed. The measures used in our
analyses were based on statepolicies reportedin
1993, the yearduringwhich most NELS students
graduatedfrom high school.
304
In this studywe includedan indicatorof states'
academic course requirementsfor high school
graduation,the oldest strategy for raising academic standardsby requiring students to take
more courses. In the NLSS questionnaire,states
were asked to report the number of Carnegie
units in various subjects that students were requiredto completeto be eligible for a high school
diploma. The variableused in this analysis was
the total numberof credits requiredin the four
core academicsubjectsof English, social studies,
mathematicsand science. (See AppendixB for a
full descriptionof the statevariables.)Only three
states (Colorado,Massachusetts,andWyoming)
reportedsetting no course requirementsfor high
school graduation,while the remainingstatesrequiredan averageof 9.96 creditsin these subjects
to earna diploma.
Our measure of the extensiveness of states'
testing programsin 1993 was based on their reportsof the gradelevels andmajoracademicsubjects in which mandatedtests were administered
to studentsduringhigh school. Only seven states
reported no mandated testing of high school
studentsin the major subjects of English, social
studies/history,mathematics, and science. The
remainingstatesgave on averagefourteststo high
school students,althoughtwo states (Minnesota
andVirginia)reportedtesting studentsin all four
subjectseveryyear.The extensivenessof a state's
testing programis an indicatorof whetherexternal examinations were used on a regular basis
to monitorstudents'progressthroughthe established curriculum,usually with the intention of
raising overall levels of achievement.
Although most states tested high school students, they varied in the extent to which performance on those tests carriedmeaningfulconsequences for studentsor schools. Ourmeasureof
consequences for studentsbased on test performance was the sum of states' reportsof whether
test scoreswererecommendedor requiredforpurposes such as placementin remedialor advanced
placementprograms,promotionto the next grade,
or awardof a high school diploma. Almost two
thirdsof the states had guidelines or mandatory
policies specifyinghow test scoresshouldbe used
to determinesome aspect of students' academic
program or success. Those states with such
policies linked test scores to an average of three
or four consequences for students. Fewer states
linked students'performanceon mandatedtests
Raising the Bar and Equity?
to rewardsandsanctionsfor schools in 1993. The
survey asked abouteight types of consequences,
such as financialrewardsfor meeting standards
or sanctionslike loss of accreditationfor failure
to do so. Over two thirds of the states reported
thatthey eitherdid not set performancestandards
or did not provide incentives for meeting those
standards.The remainingthirdof the stateslinked
aggregatetest scoresto an averageof threeor four
consequencesfor schools.
In preliminaryanalyses, the four measuresof
states' policies appearedto reflectdistinctstrategies for increasingstandardsand accountability.
The four measures of state policies were only
moderately related to each other with correlations all less than .36. The strongestcorrelation
reflected that the number of consequences for
schools and for students linked to test performance was relatedto stateshaving a testing program.However, the extensiveness of testing and
how results were used to determinesanctionsor
rewardstendedto be unrelated.
Analysis technique
The questionsof whetherstatetestingpolicies
were relatedto students'mathematicscourses in
high school requireda multilevel analytic strategy. We were concernednot only with variation
in students' mathematics course taking across
states with differingpolicies (directeffects), but
also with whether the associations of students'
outcomes with their social backgroundsvaried
across states (interaction effects). A common
techniquefor analyzinghierarchicaldata(in this
case, students nested within states) and crosslevel effects is HLM, which allows simultaneous considerationof factors from two levels of
analysis (Bryk & Raudenbush, 1992; Raudenbush & Bryk, 1986).4
The same student and state policy variables
were used for analyses of students' freshman
mathematicscourse level and the numberof advancedmathematicscreditsearned,except freshman course level was also used to predict the
lateroutcome.The student-levelmodel is shown
in Equation1, where ij was the value for a given
studentin a given stateandBkjwas the coefficient
for students'SES, race and ethnicity,or the control variablesin each state. The effects for some
of the student-levelfactors, such as race or ethnicity, were expressedby severalcoefficients,for
example B2j for Latino/a and B3j for African-
American.The termeijwas a measureof the random error,which included unmeasuredsources
of variationin a particularstudent'soutcome. In
our analyses, all the student-levelvariableswere
centered aroundtheir grandmeans for the sample, which allowed the intercept(Boj)to be interpretedas the meanoutcomefor each stateadjusted
for the characteristics of students in that state
(Bryk, Raudenbush,& Congdon, 1996; Willms
& Raudenbush,1989).
Y, = P,j + By (SESi,)
+ B2j-3j(Race/Ethnicityij)
+ B4j-0j (Controls) + ei
(1)
Preliminaryanalysesindicatedthat,in oursample, the associations between the student-level
controlvariablesandmathematicscourseseither
did not vary significantly across states or those
variationswere not related to state testing policies. Either situation meant that assuming the
associations were constant across states did not
substantiallyaffect the results for SES and race
or ethnicity. Thus, for the analyses presented
here,the coefficientsfor the student-levelcontrol
variables were set to be "fixed effects" and our
statistical model assumed that the relationships
between these studentcharacteristicsandmathematics course enrollmentswere the same for all
states (Bryk & Raudenbush,1992).
The state-levelanalyses,in essence, examined
the extent to which variationin the coefficients
for the intercept,SES, andrace or ethnicitywere
related to states' graduationand accountability
policies. Equation 2 shows the general model
used for estimatingthe effects of these statepolicies.5 Each of the policy variableswas centered
aroundits grandmean, which meantthatYk0was
the averageeffect of variablek across states and
the other coefficients were adjustmentsto those
coefficients, or interactioneffects, for states that
differed in their testing policies. The effect of a
student-level variable was increased when the
coefficient for a state policy variablewas in the
same direction (plus or minus) as the intercept
for the student-levelvariableand reducedwhen
the two coefficients were in the opposite direction. The termukjwas the errorterm for estimation of the student-levelcoefficientfor each state.
Bkj = YkO + Ykl-k4(State
Policiesj) +ukj
(2)
305
Schiller and Muller
The combined HLM model is shown in
Equation 3.
Yi = [Too+ Yoi-o4(State Policiesj)+
oj]
+[710 + Y1-04(State Policiesj)+
+[2-30
+
22-34(State Policiesj)
lj]*SESi
+U23j]
* Race/Ethnicityij
+
4-100* Control Variablesi
+eij
(3)
Results
States vary in their approachesto raisinghigh
school students' academic attainmentand promoting equalityof opportunitiesfor learning,but
the extentto whichthesepolicies impactstudents'
educationalcareersis uncertain.The purposeof
our study was to examine variationacross states
adoptingdifferentstrategiesfor raisingstandards
and establishingaccountabilityin two criticalaspectsof students'mathematicscourseenrollments:
wherethey startedas freshmen,andthe amountof
advanced-levelcourseworkcompletedby graduation. The goal was to determinewhetherthese
statepolicies were relatedto students'mathematics course placementsas freshmenand theirpersistence in advancedmathematicsas well as differencesbased on SES and race or ethnicity.
Freshman Mathematics Course Placements
Theresultsfor students'freshmanmathematics
courseenrollmentsare shownin Table 1. The top
panel of the table containsthe coefficientsfor the
interceptand independentvariablesmodeled on
the statelevel. The firstcolumn shows the Level2 intercept,or averageeffect, for the student-level
variables of interest. The other four columns
show the coefficients for state policy variables.
The lower panel containsthe coefficients for the
student-levelcontrols,which were assumedto be
constantacross states.
All of the student-levelcontrolvariablesexcept
urbanicitywere significantpredictorsof students'
mathematicscourseplacementsas freshmen.Students tendedto enroll in a higher level course if
they were female, lived with bothnaturalparents,
had highermathematicsgradesin middle school,
and enrolledin Algebraas an 8th grader.6Eighth
graderswho took remedial mathematicstended
to be placed in lower level courses as freshmen,
306
even takinginto accounttheirsocial backgrounds
andmiddleschool mathematicsgrades.Freshman
courseenrollmentsappearedto have been similar
acrossurban,suburban,andrurallocations.
The first row of Table 1 indicates that states'
graduationand accountabilitypolicies were related to differences in where freshmenenter the
high school mathematics sequence. On average, freshmen tended to enroll in pre-algebra
(coded as 3). In states requiringmore academic
coursecreditsfor graduation,freshmentendedto
take slightly higher level mathematicscourses.
Although statisticallysignificant,this effect was
small at less than 7% of a course level per standarddeviation change in the numberof courses
required(.077 = .024 * 3.203). This difference,
however, was only slightly smaller than those
relatedto genderor family structure.Extensiveness of testing was also significantly related to
freshman course enrollments, with students in
states with more extensive testing tendingto enroll in slightlylower level freshmanmathematics
courses (-.059 = -.017 * 3.461). Neither of the
state accountabilitypolicy variableswere significantly relatedto freshmancourse enrollments.
Extensivetestingwas also significantlyrelated
to a somewhat strongereffect of socioeconomic
status on freshmanmathematicscourse level. A
standarddeviationincreasein the numberof tests
was relatedto almosta 20%increasein the effect
of SES [.197 = (.018 * 3.461)/.314]. These results indicatethe gaps between poor andrich students were larger in states that test high school
students more frequently and in more subjects.
The strongereffect of SES in states with more
extensive testing was consistentwith analysesof
otheracademicoutcomes such as earninga high
school diploma(Muller& Schiller, 2000).
Our results identified no overall differences
based on race or ethnicity after controlling for
prioracademicperformanceand SES. However,
statepolicieswererelatedto significantdifferences
in freshman mathematics course enrollments
between African American and white students.
The tendency for African American studentsto
enroll in somewhat lower level courses comparedto similarwhites was strongerin statesrequiring more academic courses for graduation
or linking test performanceto consequences for
students. The latter policy strategy more than
doubled the effect of being African American
for each additionalconsequence.Conversely,the
TABLE 1
Effectsof State Policies on the Level of FreshmanMathematicsCourses
State Policy
Student-levelVariable
Average Effect
Intercept
Socioeconomic status
3.507***
.314***
Race/ethnicity
Latino/a
AfricanAmerican
-.030
-.074
Student-levelControl
Male
Living with both parents
Middle school mathgrades
8th-grademathclass
Remedial
Algebra/advanced
Urbanicity
Urban
Rural
* =p < .05, ** =p