Vigdor and Clotfelter 15
These residual test score gains can be considered real and attributable either to a gain from familiarity with the test or to gains due to learning more over time.
18
V. Model and Implications
To better understand the origins and implications of retaking and to consider how behavior might change under alternative policy regimes, it is helpful
to think about what goes into the decision to retake the test. We find it useful to begin by thinking of the SAT as simply a series of questions designed so that a
given student faces the same probability ρ of answering each question correctly. We
refer to ρ as true ‘‘ability,’’ although by using this term we do not mean to weigh
in on the question of what exactly the test does measure. In this case, the percentage of questions a student actually answers correctly, p, is an estimate of the true parame-
ter ρ. As described, this setup is equivalent to a series of Bernoulli trials, and the
distribution of p is given by a binomial. We extend this logic to two different kinds of ability, mathematical and verbal, with the student’s true mathematical ability
ρ
m
and true verbal ability ρ
v
. Following the binomial analogy, we assume the population distribution of these parameters falls between zero and one, with the potential that
these two ability measures may be correlated with one another. Importantly, we also assume that applicants do not know their true ability parameters, but make inferences
about them on the basis of information received through test scores and other sources.
We envision a simple admissions process in which the college’s admissions office attempts to rank its applicants according to ability, based on the point estimates
received through applicant test scores. The applicant’s objective in deciding how many times to take the test is to maximize the probability of admission, subject to
cost-related constraints. Consistent with the notion that applicant ability as measured by the SAT is not the only criterion for admission, we presume that there is no set
of SAT scores that guarantees admission.
Applicants who retake the SAT are effectively submitting multiple point estimates of their true mathematical and verbal ability. It is clear that applicants’ incentives
to retake the test will be influenced by how a college treats these multiple point estimates. Were colleges to accept only the first set of point estimates,
{ p
1 m
, p
1 v
}, then applicants would never retake the test, so long as the cost of taking the test
were positive. When colleges consider point estimates other than the first set in determining their ranking, students can be expected to retake the test whenever they
believe the benefits of doing so will exceed the costs.
The highest-score policy commonly used by college admissions offices translates into the following rule: for a student who has taken the test n times, use the maximum
that 18 percent of the population score more than 2.5 standard deviations above their individual means on the first administration. This presumption strikes us as untenable. As additional support of this evidence,
the simulation results discussed in Section VI below also suggest that selection plays at most a small role. 18. We have estimated the average expected ‘‘upward drift’’ for applicants in our sample. It is reasonable
to think that expected upward drift varies within the population. Some individuals may learn more quickly than others, or adapt their test-taking behavior more rapidly. Variation in expected upward drift could
provide another reason for individuals with identical initial test scores to make different retaking decisions. The effect of this variation would be similar to variation in prior beliefs, as described in our model.
16 The Journal of Human Resources
mathematical score, max { p
1 m
, . . . , p
n m
} and the maximum verbal score, max {p
1 v
, . . . , p
n v
} as point estimates of ρ
m
and ρ
v
. Applicants will choose to retake the test when they judge that the increase in the probability of admission associated with
expected changes in either their mathematical or verbal score is sufficiently large to justify the costs of retaking. The applicant’s problem becomes analogous to one of
optimal search with the possibility of recall Stigler 1961, DeGroot 1968. An appli- cant who faces dollar-denominated test-taking costs, including fees, opportunity
costs, and psychic costs, equal to c, places a dollar value V on admission, and has received maximum math and verbal scores of p
m
and p
v
in their previous test admin- istrations will retake the test if and only if:
2
冱
p
m
冱
p
v
V
冢
a max
{ p
m
, p
m
}, max{ p
v
, p
v
} ⫺ ap
m
, p
v
冣
f p
m
f p
v
⬎ c. where a p
m
, p
v
represents the probability of admission given test scores p
m
and p
v
, f
p
m
is the applicant-specific probability density function for math point estimates, and f p
v
is the corresponding probability density for verbal point estimates.
19
Imbed- ded in the equation is the assumption that in each test administration, the point esti-
mates p
m
and p
v
are drawn independently from their respective marginal distributions. The equation also assumes that the point estimates take on a finite number of val-
ues—a reasonable assumption, since there are only 61 unique scores on the SAT math and verbal scales.
20
If applicants knew the value of their underlying ability parameters with certainty, their optimal decision would be to continue taking the test until they had achieved
some ‘‘reservation test score.’’
21
But because we presume that students do not know their underlying ability parameter with certainty, the typical reservation test score
rule will not apply Rothschild 1974.To determine an applicant’s decision rule in this scenario, we presume that applicants receive and process information in a Bayes-
ian manner.
We begin by assuming that applicants receive prior information regarding their subjective distributions of f
ρ
m
and f ρ
v
by receiving ‘‘practice draws’’—pre–test administration Bernoulli trials that might be thought of as information contained in
school grades, previous standardized test scores, and the like. Along with the scores from their first test administration, these draws form their information set as they
decide whether to take the test a second time. Based on the information contained in their first test scores and practice draws, applicants form a posterior probability
distribution for the underlying parameters for use in their decision on whether to retake the test. Following any subsequent test administrations, applicants once again
19. We assume here that all applicants face the same acceptance probabilities, a p
m
, p
v
. The simulation we perform is unaffected if the acceptance probability surface mapped in Figure A2 shifts up or down
uniformly. To the extent, however, that the acceptance probability surface differs substantially in shape across categories of applicants, we are overlooking an important source of variation in behavior. Affirma-
tive action programs, for example, may result in a leveling up in the admission probability surface for some groups, which might in turn explain their reduced likelihood of retaking the test.
20. Scores range from 200 to 800 in increments of 10. 21. Because there are two elements to the SAT score, there would be no unconditional reservation values
of p
m
or p
v
. Rather there would be conditional reservation values: the value of p
m
that will induce an applicant to stop taking the test depends on what p
v
is and vice-versa.
Vigdor and Clotfelter 17
update their posterior probability distributions. As changes in an applicant’s beliefs about her true ability influence the probability she attaches to receiving any particular
test score, her ‘‘reservation test scores’’ may change over time. Two applicants who receive identical scores on their first test may be differentially
likely to retake the test for three basic reasons. First, they may face different costs of retaking the test. Those with part-time jobs, for instance, will tend to face higher
opportunity costs of taking a test than other applicants. Applicants may have differen- tial psychic costs of undergoing a testing procedure. Even testing fees themselves,
which are generally constant, may impose differential utility costs. Second, the value they attach to being admitted to a college may differ. These first two factors can be
consolidated into one: applicants may differ in the ratio of their test-taking costs to the benefits they attach to admission—the ratio cV. Third, their prior beliefs, based
on their practice draws, may lead them to expect different scores on their next test.
VI. Simulating Test-Taking Behavior
