Michael Baker is a professor of economics at the University of Toronto and a faculty research fellow of the NBER. Mark Stabile is an associate professor of economics at the University of Toronto and a faculty
research fellow of the NBER. Catherine Deri is an assistant professor of economics at the University of Ottawa. This is an updated and revised version of Baker, Stabile, and Deri 2001. The authors gratefully
acknowledge the research support of SSHRC Baker, Grant 410-99-0112, CIHR Baker and Stabile, Grant MOP-53133 the Institute for Clinical and Evaluative Sciences Stabile and Deri. The authors
would like to thank John Bound, John Ham, David Cutler, the referees, and seminar participants at vari- ous workshops for helpful comments. The data can be obtained through application to the Ontario
Ministry of Health and Long Term Care. [Submitted November 2002; accepted July 2003]
ISSN 022-166XE-ISSN 1548-8004 © 2004 by the Board of Regents of the University of Wisconsin System
T H E J O U R NA L O F H U M A N R E S O U R C E S
●
X X X I X
●
4
What Do Self-Reported, Objective, Measures of Health Measure?
Michael Baker Mark Stabile
Catherine Deri
A B S T R A C T
Survey reports of the incidence of chronic conditions are considered by many researchers to be more objective, and thus preferable, measures of
unobserved health status than self-assessed measures of global well being. In this paper we evaluate this hypothesis by attempting to validate these
“objective, self-reported” measures of health. Our analysis makes use of a unique data set that matches a variety of self-reports of health with respon-
dents’ medical records. We find that these measures are subject to consider- able response error resulting in large attenuation biases when they are used
as explanatory variables.
I. Introduction
The limitations of using subjective, self-reported assessments of global health andor physical capacity in empirical models of labor market behavior
are now widely recognized see Currie and Madrian 1999 for a review. The problems range from the so-called “justification hypothesis”—health problems are a socially
acceptable and convenient rationalization of absence from the labor market—to the unknown level of comparability of these subjective evaluations across individuals. As
a result, many researchers put greater stock in more objective, but still self-reported,
records of specific illnesses or information on subsequent mortality as proxies for health Bound and Burkhauser 1999. The argument is that these measures are
based on very specific questions, which by their nature constrain the likelihood that respondents rationalize their own behavior through their answers.
The enthusiasm for these objective measures has influenced survey design. For example, questions capturing the incidence of specific ailments in the Health and
Retirement Study HRS have this source, as the following passage from the survey documentation reveals.
. . . the Health Conditions and Health Status Working Group was persuaded that the most important dimension to measure was functional health, not disease epi-
demiology. Some potential users of the data took a different view: that a basic inventory of important medical conditions would not only satisfy the demands of
an important class of analytic users, but also might be less susceptible to misre- porting because of the causality problem involving the relation of health to work
The Early HRS and AHEAD Surveys, Revised August 4, 1999.
Implicit in arguments for the superiority of these measures is an assumption that they can be validated. Their advantage is that they record the facts of an individual’s cur-
rent andor past medical conditions rather than their opinions on their physical well- being. Facts can be checked.
In this paper we attempt to validate the self-reported measures of specific ailments contained in the 199697 Canadian National Population Health Survey NPHS. This
is a nationally representative survey of health including measures of 1 self-reported global health, 2 specific work and activity limitations, and 3 the self-reported inci-
dence of specific ailments. The validation exercise is made possible by a link between the data for respondents in the province of Ontario and diagnosistreatment informa-
tion following the International Classification of Diseases standard—9 revision, ICD-9 taken from Ontario Health Insurance Plan OHIP records for these same indi-
viduals from the survey and preceding five years. OHIP is a public healthcare program financed out of tax revenues, which covers all individuals in the province subject to
certain residency requirements. Because private alternatives to public health are either prohibited by law or are relatively very expensive for example, going to U.S. health-
care providers
1
, these OHIP records should provide a very complete record of the diagnoses and treatments of these individuals over the period.
We investigate the “measurement error” in the NPHS variables, on the assumption that the medical records are the “truth” that the questions attempt to capture. We begin
quantifying the incidence of “false positives” and “false negatives” in the self-reports of the major disease categories. For many of the diseases we find that more than 50
percent of the individuals who have a diagnosis in the OHIP data fail to report having the disease. Similarly, frequently more than 50 percent of individuals who report hav-
ing a disease have no corresponding OHIP record. We next conduct a series of exer- cises to bound the measurement error and show that while there is likely some error
in the OHIP data the majority of the error that we find comes from the NPHS.
1. The 199697 NPHS asked the respondents whether that had received health care in the United States in the past 12 months. Less than 1 percent of the sample the sample is Canada-wide reported receiving such
care.
The Journal of Human Resources 1068
We then quantify the measurement error using tools familiar from studies of measurement error in labor market data. Estimates of the proportional bias when the
self-reports of ailments are used as the sole explanatory variable in a regression average around 0.5 and range from 0.2 to as large as 0.9. These estimates are larger
than comparable calculations in past studies for labor earnings, but of the same mag- nitude as estimates for hourly earnings. It is important to note, however, that
researchers are typically not interested in the effects of specific ailments on labor mar- ket activity, but instead the effects of some underlying work capacity that presumably
both objective and subjective self-reported health variables measure with error.
In the final sections we test and find evidence for the “justification hypothesis” and investigate how the error in self-reported health varies with the intensity of a condi-
tion. We also formally test whether the information provided by the self-reports and OHIP records is identical, conditional on observable characteristics.
II. The Data
