Directory UMM :Data Elmu:jurnal:J-a:Journal of Economic Psychology:Vol21.Issue3.Jun2000:
Journal of Economic Psychology 21 (2000) 305±317
www.elsevier.com/locate/joep
Testing for consistency in willingness to pay experiments
Mandy Ryan *, Fernando San Miguel
Health Economics Research Unit, Department of Public Health, University Medical Buildings, Foresterhill,
Aberdeen AB25 2ZD, UK
Received 27 November 1998; received in revised form 20 June 1999; accepted 13 November 1999
Abstract
Given the increased use of willingness to pay (WTP) experiments to elicit the economic
value of goods, it is becoming increasingly important to assess the validity of the instrument.
Given the problems of testing for external validity in the absence of market, other methods
must be developed to test the validity of WTP responses. This paper develops a simple test of
consistency in WTP experiments which is based on the theoretical basis of the technique ± if
commodity A is preferred to B, then individuals should be willing to pay more for A than B.
The test is applied to elicit womenÕs preferences for two alternative treatments for menorrhagia: conservative treatment versus hysterectomy. Thirty percent of respondents failed the
consistency test. Cost-based responses were found to partly explain inconsistent responses.
This simple test highlights potential problems when using WTP experiments within a costbene®t analysis framework. Possible solutions to avoid Ôcost-basedÕ WTP responses are suggested. Ó 2000 Elsevier Science B.V. All rights reserved.
PsycINFO classi®cation: 3920
JEL classi®cation: C21; C42; C52; D12
Keywords: Willingness to pay; Consistency
*
Corresponding author. Tel.: +44-1224-681818 ext 54965; fax: +44-1224-662994.
E-mail address: [email protected] (M. Ryan).
0167-4870/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 7 - 4 8 7 0 ( 0 0 ) 0 0 0 0 6 - 4
306
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
1. Introduction
Willingness to pay (WTP) experiments have been widely used in environmental economics, and are being increasingly used in health economics,
as a method for valuing the bene®ts of non-marketed commodities (Cummings, Brookshire & Schulze, 1986; Mitchell & Carson, 1989; Carson,
Wright, Alberini, Carson & Flores 1994; Diener, OÕBrien & Gafni, 1998).
Using a survey design, individuals are presented with a hypothetical scenario and asked, in this scenario, for their maximum WTP for the good
concerned. 1
Given that WTP experiments are based on responses to hypothetical
questions, tests of their validity must be carried out. Ideally such tests
would involve testing the external validity of responses, i.e. the extent to
which individuals behave as they say they would. However, in some areas,
such as the UK health care system, such tests are dicult to carry out
since individuals do not pay at the point of consumption. There is no
market in which they can reveal their true maximum WTP. Studies applying WTP experiments to health care have tested for theoretical validity
(the extent to which results are consistent with economic theory, or,
sometimes, a priori expectation more generally) and convergent validity
(the extent to which values elicited using the WTP instrument are similar
to values estimated using other techniques) (Johannesson, Jonsson &
Borgquist, 1991; Johannesson, Johansson, Kristrom & Gerdtham, 1993;
OÕBrien & Viramontes, 1994; Donaldson, Shackley, Abdalla & Miedzybrodzka, 1995; Donaldson, Shackley & Abdalla, 1997; Ryan, Ratclie &
Tucker, 1997).
This paper adds to the literature on the applicability of WTP experiments
by looking at the consistency of responses. Given that WTP is based on the
value of a commodity, individuals should be willing to pay more for preferred
goods. Thus, one way of measuring the consistency of WTP experiments is to
ask respondents their preference for the commodities being valued and then
compare such preferences with their stated WTP. A respondent that gives a
smaller WTP value for their preferred option or greater WTP value for their
less preferred option may be considered inconsistent. This test is applied to
eliciting womenÕs preferences for two alternative treatments for menorrhagia:
1
Willingness to pay experiments are also referred to in the literature as contingent valuation studies.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
307
conservative treatment versus hysterectomy. 2 Following this, possible reasons for inconsistent responses are explored, as are the possible implications
of including inconsistent responses to estimate welfare changes. Finally,
possible methods to overcome cost-based WTP responses are suggested.
2. Methods
2.1. Subjects and settings
Following a pilot study the main questionnaire was mailed to a sample of
146 women in the Department of Public Health and the Royal Maternity
Hospital in Aberdeen. Due to time limitations, no reminders were sent.
2.2. Willingness to pay question
Women were provided with information on menorrhagia and two alternative treatments ± hysterectomy and conservative treatment (Pinion et al.,
1994; Cameron et al., 1996). Respondents were reminded that any money
they spent on menorrhagia treatment would not be available to spend on
other goods or services. Following this, the payment card (PC) WTP technique was used to elicit maximum WTP in an experimental situation. Respondents were presented with a range of monetary values and asked to circle
the maximum amount they would be willing to pay for each treatment. Based
on the results from the pilot study, the PC ranged from £0 to £5000. If individuals answered £5000 they were asked to state the maximum amount they
would be willing to pay. They were also asked to give reasons for their answers.
Information was also collected on time to complete the questionnaire, ease
of completion (with possible responses on a 1±5 scale, where 1 was very easy
and 5 very dicult) and socio-economic characteristics of respondents.
2
Menorrhagia, or excessive menstrual bleeding, aects approximately 22% of healthy women. Due to
the failure of drug treatments for the condition, 60% of general practitioner referrals for menorrhagia are
treated by hysterectomy within ®ve years. Recently, new minimal access surgical alternatives to
hysterectomy, known as conservative treatment, have been developed (Pinion, Parkin, Abramovich, Naki,
Alexander, Russell & Kitchener, 1994; Cameron, Mollison, Pinion, Atherton-Naji, Buckingham &
Torgerson, 1996).
308
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
2.3. Testing for consistency
Following the WTP question, women were presented with a dichotomous
choice question which asked them which of the two treatments they preferred. WTP responses were recoded into a nominally scaled variable where
1 WTPhysterectomy > WTPconservative and 2 WTPconservative > WTPhysterectomy .
(Individuals who stated WTPhysterectomy WTPconservative were not included
here.) v2 tests were carried out to see if the distribution of responses of both
the dichotomous choice question and recoded WTP were signi®cantly different across the two alternative treatments (Bland, 1995). The aim here was
to establish if the preference structure was signi®cantly different across the
two elicitation techniques.
Information on WTP and preferences was then combined to test for
consistency of responses. WTP was estimated at the individual level for both
hysterectomy and conservative treatment. Individuals should be willing to
pay more for their preferred treatment. Responses that did not satisfy this
condition were considered ÔinconsistentÕ. 3 The Fisher Exact Test and
McNemar Test were used to test for an association and agreement between
the two methods of collecting preference data (Bland, 1995). The aim here
was to test whether the implied preference structure was a function of the
preference elicitation method.
2.4. Explaining inconsistent answers
Once inconsistent respondents are identi®ed, there is the question of what
to do with them. It may be argued that they should be dropped from the
analysis since these respondents do not understand the questions or have not
taken the exercise seriously. However, Sen (1993) argues that there may be
rational reasons for inconsistent responses. In this study an attempt was
made to explore the possible reasons for `inconsistent' responses using probit
regression techniques. The dependent variable was whether or not the respondent was consistent (0 inconsistent, 1 consistent). The independent
variables were: responses to the questions asking individuals reasons for their
WTP responses; information on ease of completion (indicated by responses
3
Responses for which WTP for both treatments were equal were considered ÔconsistentÕ. This was based
on the assumption that although some people may prefer one of the treatments, their preference may not
be strong enough to pay more for their preferred option.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
309
to the ease of completion question and time to complete the questionnaire)
and the professional background of respondents (as a proxy for the information that respondents had about the treatments, and taking a value of ``1''
for medical sta and ``0'' otherwise). A general to speci®c approach was
adopted with explanatory variables being excluded by backward elimination
if they had a signi®cance level of less than 0.05.
2.5. Implications of including inconsistent respondents
Following this, the in¯uence of including inconsistent responses is considered. WTP is ®rst estimated for the complete sample (including consistent
and inconsistent responses), and then just for consistent respondents. It was
assumed that WTP was measured on an interval scale, and the t-test was used
to see if WTP estimates were signi®cantly different across the two groups.
Ninety-®ve percent con®dence intervals were also estimated.
3. Results
There was evidence in the pilot study of inconsistent responses, with individuals willing to pay more for their less preferred option. Analysis of the
questions asking individuals the reasons for their answers indicated that respondents were providing cost-based responses. That is, whilst they preferred
conservative treatment, they perceived hysterectomy to be more expensive
and hence stated a greater WTP for hysterectomy. In an attempt to overcome
cost-based WTP values in the main study a distinction was made between
cost and value. Individuals were told:
to see why ®nding out your maximum willingness to pay shows the value you place on something imagine you are going to an auction to buy a
car. Having considered all the features of the car (e.g. make, model, colour) you will then decide your maximum willingness to pay (or value)
for the car. If you were willing to pay £2000, this is the value to you
of the car. If you only pay £1500, then the cost is smaller than the value.
Thus, the value you place on the car is not the same as the cost. The
same applies to health care interventions. That is, the value may be different from the cost. So, measuring the maximum willingness to pay
that people place for some goods is simply a way of measuring the bene®ts for them derived from the use of these goods.
310
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Of the 146 questionnaires distributed, 75 (51% response rate) were returned. Responses to the WTP and dichotomous choice questions are shown
in Table 1. Responses to the WTP question suggest that the majority of respondents prefer hysterectomy, and that the distribution of preferences
across the two treatments is signi®cantly dierent (v2 12:302, P 0.001). In
contrast, the results from the dichotomous choice question indicate that the
majority of respondents prefer conservative treatment. This distribution of
preferences across the two treatments is again signi®cantly different, though
in the opposite direction (v2 12:552, P 0:001). Thus, the WTP and dichotomous choice questions imply different preference structures. These results were con®rmed when the WTP and dichotomous choice questions were
considered together ± 41% of respondents who preferred conservative surgery
were willing to pay more for hysterectomy. 4 Both the Fisher Exact Test
(P 0:018) and McNemar Test (P 0:001) indicated that the implied
preference structure was a function of how preferences are elicited.
A large number of inconsistent respondents gave cost as a reason for their
WTP ± 60% of inconsistent respondents mentioned cost compared to 17% of
consistent respondents. From the regression analysis, cost and ease of completion were signi®cant predictors of inconsistent responses (Table 2). The
probability of giving inconsistent answers increased as people mentioned cost
as a reason for their WTP answers. People that found the exercise more
dicult were more likely to provide consistent answers. This may re¯ect the
fact that those individuals who found the WTP more dicult were more
likely to have thought about it. The insigni®cance of the attribute ``occupation'' could be due to sample size limitations and to the lack of variation in
the attribute since most of the respondents were working at the Maternity
Hospital in Aberdeen.
Table 3 compares WTP estimates for the total sample with consistent respondents. For the total sample, those preferring conservative treatment are
willing to pay £2031 for conservative treatment and £2290 hysterectomy.
Whilst this group are willing to pay more for their least preferred option, the
dierence is not statistically signi®cant (t ÿ0:930; P > 0:15). Consistent
respondents are willing to pay more for the treatment they prefer ± those who
4
In this study 19 individuals preferred hysterectomy to conservative treatment ± ®ve of these were
willing to pay equal amounts for hysterectomy and conservative treatment and 14 were willing to pay more
for their preferred option. Whilst signi®cantly less of these ÔconsistentÕ responses stated that ÔcostÕ was a
reason for their expressed WTP, it is possible that some of these responses were cost-based. The
consistency test applied here would not pick this up (since their preferred option was cheaper).
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
311
Table 1
Preferences and WTP for hysterectomy (WTPH) and conservative surgery (WTPC)
Dichotomous
choicea
WTPC >
WTPH
WTPC
WTPH
WTPC <
WTPH
Cons Hyst
(48.64%)
Hyst Cons
(19.25%)
Not de®nedb
(8.11%)
10
(19%)
0
18
(35%)
5
(26%)
3
(37%)
19
(41%)
14
(74%)
1
(13%)
26
34
Total
0
10
2
v (WTP)
Total
v2 (choice)
12.552
P 0:001
4
(50%)
48
(100%)
19
(100%)
8
(100%)
5
76
Missing
1
(5%)
0
12.302
(0.001)
0.018
0.001
FisherÕs Exact Test
McNemar Test
a
`` '' means ``strictly preferred to''.
ÔNot de®nedÕ responses included seven respondents who did not express a preference and one respondent
who chose both treatments.
b
Table 2
Results from the Probit regression model explaining inconsistent responses
Variable
General model
Coecient
Cost
Occupation
Time completion
Ease completion
N
Log likelihood function
McFadden R2
Chi-squared
)0.79
0.0010
)0.0011
0.19
65
)37.02
0.075
6.19
Speci®c model
P-value
0.03
0.30
0.10
0.00
Coecient
)0.69
)
)
0.19
P-value
0.04
)
)
0.00
65
)38.79
0.03
2.64
preferred hysterectomy were willing to pay signi®cantly more to have this
treatment (£2116 compared to £847; t ÿ4:456; P 0:001) and those that
preferred conservative surgery were willing to pay signi®cantly more for this
treatment (£2571 compared to £1711; t 2:74; P 0:001). Although WTP
values are dierent between the entire sample and consistent respondents,
only WTP for hysterectomy for those preferring conservative surgery (as
expected, due to all the inconsistencies occurring in this group) is signi®cantly
dierent between the two groups (£2290 compared to 1711; t ÿ1:89;
P < 0:05).
312
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Table 3
Mean WTP values for all the sample and the sample of consistent respondents
Entire group
Consistent respondents
t-Statistic (P-value)
Prefer hysterectomy
Mean WTP
Hysterectomy (£) (95% CI)
Mean WTP
Conservative (£) (95% CI)
Strength of preference
t-Statistic (P-value)
2100
(1465.6; 2734.3)
1031
(487; 1574.9)
2.03
t ÿ3:217; P 0:005
2116
(1701.8; 2956.9)
847
(388.3; 1435)
2.55
t ÿ4:456; p 0:001
0.77
(P > 0:15)
)0.48
(P > 0:25)
±
Prefer conservative
Mean WTP
Hysterectomy (£) (95% CI)
Mean WTP
Conservative (£) (95% CI)
Strength of preference
t-Statistic (P-value)
2290
(1625.6; 2954.3)
2031
(1492.2; 2569.7)
0.88
t ÿ0:930; P > 0:15
1711
(1129.9; 2334.3)
2571
(1629.1; 3199.4)
1.39
t 2:74; P 0:001
)1.89
(P < 0:05)
0.99
(P > 0:15)
±
*
The values forming the ratio are signi®cantly dierent at the 95% level.
Signi®cantly dierent WTP values between the entire sample and the sample of consistent respondents at
the 95% level.
**
4. Discussion and conclusions
This paper found a number of important results when considering the
application of WTP experiments to assessing the economic value of a commodity. The results from the WTP experiment were dierent to those from
the dichotomous choice question. This was partly explained by cost-based
WTP responses, with 30% of respondents willing to pay less for their preferred alternative. A number of other WTP experiments have found evidence
of cost-based WTP responses. Schkade and Payne (1994), in using verbal
protocol analysis to investigate how people respond to WTP questions, found
that individuals justify their WTP responses by referring to the cost of the
commodity being valued. Donaldson et al. (1997) found evidence of costbased responses in a study looking at dierent methods of testing for cystic
®brosis. They argued that this problem may arise when open-ended questions
are used to elicit WTP values. The study presented here found evidence of
cost-based responses when using the PC.
Baron and Maxwell (1996), in a study looking at the value of public goods,
provided respondents with dierent types of costing information. They
conclude that future WTP studies should avoid providing respondents with
information on costs. The pilot study reported here indicated that even when
individuals are not provided with information on costs, they still provide
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
313
cost-based responses. Following this, the main study reported here directly
attempted to avoid cost-based responses by including a statement in the
questionnaire explaining that cost is not the same as value. Despite this, the
experiment still found evidence of cost-based responses. Further, such an
approach may have the added disadvantage that highlighting the cost-value
distinction may alert individuals to the concept of cost.
Questions are raised concerning the eect of cost-based WTP on the application of WTP experiments within a cost±bene®t analysis (CBA) framework. If we assume that inconsistent respondents have not understood the
task and hence have not answered in terms of their maximum WTP, their
responses do not re¯ect their maximum value but rather the cost of treatment. If cost is being used as a marker for expressing WTP then WTP for low
cost options may be underestimated and WTP for high cost options may be
overestimated. Inecient policy conclusions may therefore result.
Given that cost-based responses will lead to inaccurate estimates of welfare
changes, how may such responses be avoided? To date it has been shown that
both the open-ended and PC WTP techniques are vulnerable to such responses. Would the referendum style WTP approach also be subject to such
biases? 5 This method was proposed by the US National Oceanographic and
Atmospheric Administration (NOAA) Panel on contingent valuation as the
method of choice in WTP studies of the environment (Arrow, Solow, Portney, Leamer, Radner & Schuman, 1993). 6 This recommendation was based
on the premise that this technique would not be subject to biases, though this
argument has since been challenged (Diamond & Hausman, 1994). Costbased responses may still result from the referendum approach. For example,
if respondents think the commodity being valued costs less than the bid
amount they are presented with, they may be more likely to respond negatively to the bid amount presented to them. Similarly, if they think the
commodity being valued costs more than the bid amount they are presented
with, they may be more likely to respond positively to the bid amount presented to them. This behaviour would be consistent with the Ôfair priceÕ explanation for WTP responses, i.e. people like to pay what they believe it
5
Using this approach, individuals are asked whether they would pay a speci®ed amount for a given
commodity, with possible responses usually being ÔyesÕ or ÔnoÕ. The bid amount is varied across
respondents and the only information obtained from each individual respondent is whether her maximum
WTP is above or below the bid oered (Hanemann, 1984).
6
The NOAA panel was a group of ÔexpertsÕ who oered guidelines on the use of WTP for
environmental resource damage assessments.
314
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
costs. They do not want to be exploited by paying more than the commodity
costs, nor to exploit by paying less than the commodity costs. There is evidence of this Ôfair-priceÕ behaviour in the provision of both private and public
goods (Kahneman, Knetsch & Thaler, 1986; Winer, 1986; Green, Kahneman
& Kunreuther, 1994).
One way to attempt to overcome cost-based responses is to focus on the
bene®ts from the commodities being valued (Baron & Maxwell, 1996). The
marginal WTP approach may be used to do this (Donaldson et al., 1997;
Donaldson, Hundley & Mapp, 1998). Here individuals are ®rst asked what
option or treatment they prefer. They are then asked their maximum WTP to
have their preferred option over their less preferred. A second alternative is
the use of conjoint analysis (CA) discrete choice models to estimate WTP.
Using CA respondents are presented with numerous discrete choices, which
dier in terms of the attributes of the service, and, for each choice, they are
asked to choose their most preferred. By estimating a dierence model, and
including cost as an attribute, marginal WTP values for the change from one
treatment to another can be obtained (Ryan & Hughes, 1997). Future work
should explore whether cost-based responses emerge when these instruments
are employed.
Even if the potential problem of cost-based responding in WTP experiments is successfully overcome, research has indicated a number of other
problems with using this technique to elicit the economic value of a good.
These include the insensitivity of WTP estimates to factors that might be
expected to in¯uence such responses (Kahneman, 1986; Kahneman &
Knetsch, 1992; Diamond & Hausman, 1994; Irwin & Spira, 1997); the sensitivity of WTP estimates to factors which are theoretically irrelevant to the
commodity being valued (Bishop, Heberlein & Kealy, 1983; Sellar, Chavas &
Stoll, 1985; Cummings et al., 1986; Peterson, Driver & Gregory, 1988;
Johnson, Bregenzer & Shelby, 1990; Samples & Holier, 1990; McFadden &
Leonard, 1992; Boyle, Johnson, McCollum, Desvousges, Dunford & Hudson, 1993; Irwin, Slovick, Lichenstein & Mclelland, 1993; Kealy & Turner,
1993; McFadden, 1994; Ready, Buzby & Hu, 1996; Kramer & Mercer, 1997);
the presence of strategic behaviour (Bohm, 1972; Cummings et al., 1986;
Mitchell & Carson, 1989; Posavac, 1998); and the extent to which values exist
for unfamiliar goods (Fischo, 1991; Payne & Bettman, 1993).
In conclusion, this paper developed a method to test for the consistency of
responses in WTP experiments ± if commodity A is preferred to B, then individuals should be willing to pay more for A than B. Thirty percent of respondents fail to provide consistent responses. One reason for such
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
315
inconsistencies was cost-based responses. These ®ndings are consistent with
other studies in the WTP literature. The results are important when considering the application of WTP experiments both in health care, and elsewhere. Whilst economists argue the case for the use of WTP on the grounds
that the instrument provides a cardinal measure of preferences, the evidence
presented here suggests that the technique may not even provide an ordinal
measure of preferences. Possible solutions to avoid cost-based WTP responses include the marginal WTP approach and CA. Future work should
explore the consistency of these approaches, as well as the validity of WTP
experiments more generally.
Acknowledgements
The authors would like to thank Emma McIntosh and two anonymous
referees for comments on this paper. Financial support from the Medical
Research Council (MRC), the Chief Scientist Oce of the Scottish Oce
Department of Health (SODOH), and the government of Navarre are acknowledged. The views expressed in the paper are those of the authors.
References
Arrow, K., Solow, R., Portney, P., Leamer, E., Radner, R., & Schuman, H. (1993). Report of the NOAA
panel of contingent valuation, Washington, DC. Federal Register, 10, 4601±4614.
Baron, J., & Maxwell, N. (1996). Cost of public goods aects willingness to pay for them. Journal of
Behavioural Decision Making, 9, 173±183.
Bland, M. (1995). An introduction to medical statistics. Oxford: Oxford University Press.
Bishop, R., Heberlein, T., & Kealy, M. (1983). Contingent valuation of environmental assets:
Comparisons with a simulated market. Natural Resources Journal, 23, 619±633.
Bohm, P. (1972). Estimating demand for public goods: An experiment. European Economic Review, 3,
111±130.
Boyle, K., Johnson, F., McCollum, D., Desvousges, W., Dunford, R., & Hudson, S. (1993). Valuing public
goods: Discrete versus continuous contingent-valuation responses. Department of Resource Economics,
Working Paper, University of Maine.
Cameron, I. M., Mollison, J., Pinion, S. B., Atherton-Naji, A., Buckingham, K., & Torgerson, D. (1996).
A cost comparison of hysterectomy and hysteroscopic surgery for the treatment of menorrhagia.
European Journal of Obstetrics and Gynaecology and Reproductive Biology, 70, 87±92.
Carson, R., Wright, J., Alberini, A., Carson, N., & Flores, N. (1994). A bibliography of contingent
valuation studies and papers. La Jolla, CA: Natural Resource Damage Assessment.
Cummings, R. G., Brookshire, D. S., & Schulze, W. D. (1986). Valuing environmental goods: A state of the
arts assessment of the contingent method. Totowa: Rowman and Allanheld.
316
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Diamond, P., & Hausman, J. (1994). Contingent valuation: Is some number better than no number?
Journal of Economic Perspectives, 8, 45±64.
Diener, A., O'Brien, B., & Gafni, A. (1998). Health care contingent valuation studies: A review and
classi®cation of the literature. Health Economics, 7, 313±326.
Donaldson, C., Shackley, P., Abdalla, M., & Miedzybrodzka, Z. (1995). Willingness to pay for antenatal
carrier screening for cystic ®brosis. Health Economics, 4, 439±452.
Donaldson, C., Shackley, P., & Abdalla, M. (1997). Using willingness to pay to value close substitutes:
Carrier screening for cystic ®brosis revisited. Health Economics, 6, 145±159.
Donaldson, C., Hundley, V., & Mapp, T. (1998). Willingness to pay: A method for measuring preferences
for maternity care? Birth, 25, 32±39.
Fischo, B. (1991). Value elicitation: Is there anything in there? Americam Psychologist, 46, 835±847.
Green, D., Kahneman, D., & Kunreuther, H. (1994). How the scope and method of public funding aects
willingness to pay for public goods. Public Opinion Quarterly, 58, 49±67.
Hanemann, W. (1984). Welfare evaluations in contingent valuation experiments with discrete responses:
Reply. American Journal of Agricultural Economics, 69, 332±341.
Irwin, J. R., Slovic, P., Lichenstein, S., & McClelland, G. H. (1993). Preference reversals and
measurements of environmental values. Journal of Risk and Uncertainty, 6, 5±18.
Irwin, J. R., & Spira, J. S. (1997). Anomalies in the values for consumer goods with environmental
attributes. Journal of Consumer Psychology, 6, 339±363.
Johannesson, M., Jonsson, B., & Borgquist, B. (1991). Willingness to pay for antihypertensive therapy.
Results of a Swedish pilot study. Journal of Health Economics, 10, 461±474.
Johannesson, M., Johansson, P.-O., Kristrom, B., & Gerdtham, U.-G. (1993). Willingness to pay for
antihypertensive therapy ± further results. Journal of Health Economics, 12, 95±108.
Johnson, R. L., Bregenzer, N. S., & Shelby, B. (1990). Contingent valuation question formats:
Dichotomous choice versus open-ended responses. In R. L. Johnson, & G. V. Johnson, Economic
valuation of natural resources: Issues, theory and applications (pp. 193±204). Boulder, CO: Westview
Press.
Kahneman, D. (1986). Comments on the contingent valuation method. In R. G. Cummings, D. S.
Brookshire, & W. D. Schulze, Valuing environmental goods: An assessment of the contingent valuation
method. Savage, MD: Rowman and Little®eld.
Kahneman, D., Knetsch, J., & Thaler, R. (1986). Fairness as a constraint on pro®t seeking. Entitlements in
the market. American Economic Review, 76, 728±741.
Kahneman, D., & Knetsch, J. L. (1992). Valuing public goods: The purchase of moral satisfaction. Journal
of Environmental and Management, 22, 57±70.
Kealy, M., & Turner, R. (1993). A test of equality of closed-ended and open-ended contingent valuations.
American Journal of Agricultural Economics, 75, 321±331.
Kramer, R. A., & Mercer, D. E. (1997). Valuing a global environmental good: US residents' willingness to
pay to protect tropical rain forests. Land Economics, 73, 196±210.
McFadden, D. (1994). Contingent valuation and social choice. American Journal of Agricultural
Economics, 76, 689±708.
McFadden, D., & Leonard, G. (1992). Issues in the contingent valuation of environmental goods:
Methodologies for data collection and analysis. Berkeley, CA: University of California.
Mitchell, R. C., & Carson, R. T. (1989). Using surveys to value public goods. Washington, DC: Resources
for the Future.
O'Brien, B., & Viramontes, J. (1994). A valid and reliable measure of health state preference? Medical
Decision Making, 14, 289±297.
Payne, J. W., & Bettman, J. R. (1993). The adaptive decision maker. New York: Cambridge University
Press.
Peterson, G., Driver, B., & Gregory, R. (1988). Amenity resource valuation: Integrating economics with
other disciplines. State College, PA: Venture publishing.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
317
Posavac, S. S. (1998). Strategic overbidding in contingent valuation: stated economic value of public goods
varies according to consumersÕ expectations of funding source. Journal of Economic Psychology, 19,
205±214.
Pinion, S. B., Parkin, D. E., Abramovich, D. R., Naki, A., Alexander, D., Russell, I. T., & Kitchener, H.
C. (1994). Randomised trial of hysterectomy, endometrial laser ablation and transcervical endometrial
resection for dysfunctional uterine bleeding. British Medical Journal, 309, 979±983.
Ready, R. C., Buzby, J. C., & Hu, D. (1996). Dierences between continuous and discrete contingent value
estimates. Land Economics, 72, 397±411.
Ryan, M., & Hughes, J. (1997). Using conjoint analysis to assess womenÕs preferences for miscarriage
management. Health Economics, 6, 261±273.
Ryan, M., Ratclie, J., & Tucker, J. (1997). Using willingness to pay to value alternative models of
antenatal care. Social Science and Medicine, 44, 371±380.
Samples, K. C., & Holier, J. R. (1990). Contingent valuation of wildlife resources in the presence of
substitutes and compliments. In R. L. Johnston, & G. V. Johnston, Economic valuation of natural
resources: Issues, theory, and applications. Boulder, CO: Westview Press.
Schkade, D., & Payne, J. (1994). How people respond to contingent valuation questions: a verbal protocol
analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics and
Management, 26, 88±109.
Sellar, C., Chavas, J. P., & Stoll, J. R. (1985). Validation of empirical measures of welfare change: A
comparison of nonmarket techniques. Land Economics, 61, 156±175.
Sen, A. (1993). Internal consistency of choice. Econometrica, 61, 495±521.
Winer, R. (1986). A reference price model of brand choice for frequently purchased products. Journal of
Consumer Research, 13, 250±256.
www.elsevier.com/locate/joep
Testing for consistency in willingness to pay experiments
Mandy Ryan *, Fernando San Miguel
Health Economics Research Unit, Department of Public Health, University Medical Buildings, Foresterhill,
Aberdeen AB25 2ZD, UK
Received 27 November 1998; received in revised form 20 June 1999; accepted 13 November 1999
Abstract
Given the increased use of willingness to pay (WTP) experiments to elicit the economic
value of goods, it is becoming increasingly important to assess the validity of the instrument.
Given the problems of testing for external validity in the absence of market, other methods
must be developed to test the validity of WTP responses. This paper develops a simple test of
consistency in WTP experiments which is based on the theoretical basis of the technique ± if
commodity A is preferred to B, then individuals should be willing to pay more for A than B.
The test is applied to elicit womenÕs preferences for two alternative treatments for menorrhagia: conservative treatment versus hysterectomy. Thirty percent of respondents failed the
consistency test. Cost-based responses were found to partly explain inconsistent responses.
This simple test highlights potential problems when using WTP experiments within a costbene®t analysis framework. Possible solutions to avoid Ôcost-basedÕ WTP responses are suggested. Ó 2000 Elsevier Science B.V. All rights reserved.
PsycINFO classi®cation: 3920
JEL classi®cation: C21; C42; C52; D12
Keywords: Willingness to pay; Consistency
*
Corresponding author. Tel.: +44-1224-681818 ext 54965; fax: +44-1224-662994.
E-mail address: [email protected] (M. Ryan).
0167-4870/00/$ - see front matter Ó 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 7 - 4 8 7 0 ( 0 0 ) 0 0 0 0 6 - 4
306
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
1. Introduction
Willingness to pay (WTP) experiments have been widely used in environmental economics, and are being increasingly used in health economics,
as a method for valuing the bene®ts of non-marketed commodities (Cummings, Brookshire & Schulze, 1986; Mitchell & Carson, 1989; Carson,
Wright, Alberini, Carson & Flores 1994; Diener, OÕBrien & Gafni, 1998).
Using a survey design, individuals are presented with a hypothetical scenario and asked, in this scenario, for their maximum WTP for the good
concerned. 1
Given that WTP experiments are based on responses to hypothetical
questions, tests of their validity must be carried out. Ideally such tests
would involve testing the external validity of responses, i.e. the extent to
which individuals behave as they say they would. However, in some areas,
such as the UK health care system, such tests are dicult to carry out
since individuals do not pay at the point of consumption. There is no
market in which they can reveal their true maximum WTP. Studies applying WTP experiments to health care have tested for theoretical validity
(the extent to which results are consistent with economic theory, or,
sometimes, a priori expectation more generally) and convergent validity
(the extent to which values elicited using the WTP instrument are similar
to values estimated using other techniques) (Johannesson, Jonsson &
Borgquist, 1991; Johannesson, Johansson, Kristrom & Gerdtham, 1993;
OÕBrien & Viramontes, 1994; Donaldson, Shackley, Abdalla & Miedzybrodzka, 1995; Donaldson, Shackley & Abdalla, 1997; Ryan, Ratclie &
Tucker, 1997).
This paper adds to the literature on the applicability of WTP experiments
by looking at the consistency of responses. Given that WTP is based on the
value of a commodity, individuals should be willing to pay more for preferred
goods. Thus, one way of measuring the consistency of WTP experiments is to
ask respondents their preference for the commodities being valued and then
compare such preferences with their stated WTP. A respondent that gives a
smaller WTP value for their preferred option or greater WTP value for their
less preferred option may be considered inconsistent. This test is applied to
eliciting womenÕs preferences for two alternative treatments for menorrhagia:
1
Willingness to pay experiments are also referred to in the literature as contingent valuation studies.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
307
conservative treatment versus hysterectomy. 2 Following this, possible reasons for inconsistent responses are explored, as are the possible implications
of including inconsistent responses to estimate welfare changes. Finally,
possible methods to overcome cost-based WTP responses are suggested.
2. Methods
2.1. Subjects and settings
Following a pilot study the main questionnaire was mailed to a sample of
146 women in the Department of Public Health and the Royal Maternity
Hospital in Aberdeen. Due to time limitations, no reminders were sent.
2.2. Willingness to pay question
Women were provided with information on menorrhagia and two alternative treatments ± hysterectomy and conservative treatment (Pinion et al.,
1994; Cameron et al., 1996). Respondents were reminded that any money
they spent on menorrhagia treatment would not be available to spend on
other goods or services. Following this, the payment card (PC) WTP technique was used to elicit maximum WTP in an experimental situation. Respondents were presented with a range of monetary values and asked to circle
the maximum amount they would be willing to pay for each treatment. Based
on the results from the pilot study, the PC ranged from £0 to £5000. If individuals answered £5000 they were asked to state the maximum amount they
would be willing to pay. They were also asked to give reasons for their answers.
Information was also collected on time to complete the questionnaire, ease
of completion (with possible responses on a 1±5 scale, where 1 was very easy
and 5 very dicult) and socio-economic characteristics of respondents.
2
Menorrhagia, or excessive menstrual bleeding, aects approximately 22% of healthy women. Due to
the failure of drug treatments for the condition, 60% of general practitioner referrals for menorrhagia are
treated by hysterectomy within ®ve years. Recently, new minimal access surgical alternatives to
hysterectomy, known as conservative treatment, have been developed (Pinion, Parkin, Abramovich, Naki,
Alexander, Russell & Kitchener, 1994; Cameron, Mollison, Pinion, Atherton-Naji, Buckingham &
Torgerson, 1996).
308
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
2.3. Testing for consistency
Following the WTP question, women were presented with a dichotomous
choice question which asked them which of the two treatments they preferred. WTP responses were recoded into a nominally scaled variable where
1 WTPhysterectomy > WTPconservative and 2 WTPconservative > WTPhysterectomy .
(Individuals who stated WTPhysterectomy WTPconservative were not included
here.) v2 tests were carried out to see if the distribution of responses of both
the dichotomous choice question and recoded WTP were signi®cantly different across the two alternative treatments (Bland, 1995). The aim here was
to establish if the preference structure was signi®cantly different across the
two elicitation techniques.
Information on WTP and preferences was then combined to test for
consistency of responses. WTP was estimated at the individual level for both
hysterectomy and conservative treatment. Individuals should be willing to
pay more for their preferred treatment. Responses that did not satisfy this
condition were considered ÔinconsistentÕ. 3 The Fisher Exact Test and
McNemar Test were used to test for an association and agreement between
the two methods of collecting preference data (Bland, 1995). The aim here
was to test whether the implied preference structure was a function of the
preference elicitation method.
2.4. Explaining inconsistent answers
Once inconsistent respondents are identi®ed, there is the question of what
to do with them. It may be argued that they should be dropped from the
analysis since these respondents do not understand the questions or have not
taken the exercise seriously. However, Sen (1993) argues that there may be
rational reasons for inconsistent responses. In this study an attempt was
made to explore the possible reasons for `inconsistent' responses using probit
regression techniques. The dependent variable was whether or not the respondent was consistent (0 inconsistent, 1 consistent). The independent
variables were: responses to the questions asking individuals reasons for their
WTP responses; information on ease of completion (indicated by responses
3
Responses for which WTP for both treatments were equal were considered ÔconsistentÕ. This was based
on the assumption that although some people may prefer one of the treatments, their preference may not
be strong enough to pay more for their preferred option.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
309
to the ease of completion question and time to complete the questionnaire)
and the professional background of respondents (as a proxy for the information that respondents had about the treatments, and taking a value of ``1''
for medical sta and ``0'' otherwise). A general to speci®c approach was
adopted with explanatory variables being excluded by backward elimination
if they had a signi®cance level of less than 0.05.
2.5. Implications of including inconsistent respondents
Following this, the in¯uence of including inconsistent responses is considered. WTP is ®rst estimated for the complete sample (including consistent
and inconsistent responses), and then just for consistent respondents. It was
assumed that WTP was measured on an interval scale, and the t-test was used
to see if WTP estimates were signi®cantly different across the two groups.
Ninety-®ve percent con®dence intervals were also estimated.
3. Results
There was evidence in the pilot study of inconsistent responses, with individuals willing to pay more for their less preferred option. Analysis of the
questions asking individuals the reasons for their answers indicated that respondents were providing cost-based responses. That is, whilst they preferred
conservative treatment, they perceived hysterectomy to be more expensive
and hence stated a greater WTP for hysterectomy. In an attempt to overcome
cost-based WTP values in the main study a distinction was made between
cost and value. Individuals were told:
to see why ®nding out your maximum willingness to pay shows the value you place on something imagine you are going to an auction to buy a
car. Having considered all the features of the car (e.g. make, model, colour) you will then decide your maximum willingness to pay (or value)
for the car. If you were willing to pay £2000, this is the value to you
of the car. If you only pay £1500, then the cost is smaller than the value.
Thus, the value you place on the car is not the same as the cost. The
same applies to health care interventions. That is, the value may be different from the cost. So, measuring the maximum willingness to pay
that people place for some goods is simply a way of measuring the bene®ts for them derived from the use of these goods.
310
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Of the 146 questionnaires distributed, 75 (51% response rate) were returned. Responses to the WTP and dichotomous choice questions are shown
in Table 1. Responses to the WTP question suggest that the majority of respondents prefer hysterectomy, and that the distribution of preferences
across the two treatments is signi®cantly dierent (v2 12:302, P 0.001). In
contrast, the results from the dichotomous choice question indicate that the
majority of respondents prefer conservative treatment. This distribution of
preferences across the two treatments is again signi®cantly different, though
in the opposite direction (v2 12:552, P 0:001). Thus, the WTP and dichotomous choice questions imply different preference structures. These results were con®rmed when the WTP and dichotomous choice questions were
considered together ± 41% of respondents who preferred conservative surgery
were willing to pay more for hysterectomy. 4 Both the Fisher Exact Test
(P 0:018) and McNemar Test (P 0:001) indicated that the implied
preference structure was a function of how preferences are elicited.
A large number of inconsistent respondents gave cost as a reason for their
WTP ± 60% of inconsistent respondents mentioned cost compared to 17% of
consistent respondents. From the regression analysis, cost and ease of completion were signi®cant predictors of inconsistent responses (Table 2). The
probability of giving inconsistent answers increased as people mentioned cost
as a reason for their WTP answers. People that found the exercise more
dicult were more likely to provide consistent answers. This may re¯ect the
fact that those individuals who found the WTP more dicult were more
likely to have thought about it. The insigni®cance of the attribute ``occupation'' could be due to sample size limitations and to the lack of variation in
the attribute since most of the respondents were working at the Maternity
Hospital in Aberdeen.
Table 3 compares WTP estimates for the total sample with consistent respondents. For the total sample, those preferring conservative treatment are
willing to pay £2031 for conservative treatment and £2290 hysterectomy.
Whilst this group are willing to pay more for their least preferred option, the
dierence is not statistically signi®cant (t ÿ0:930; P > 0:15). Consistent
respondents are willing to pay more for the treatment they prefer ± those who
4
In this study 19 individuals preferred hysterectomy to conservative treatment ± ®ve of these were
willing to pay equal amounts for hysterectomy and conservative treatment and 14 were willing to pay more
for their preferred option. Whilst signi®cantly less of these ÔconsistentÕ responses stated that ÔcostÕ was a
reason for their expressed WTP, it is possible that some of these responses were cost-based. The
consistency test applied here would not pick this up (since their preferred option was cheaper).
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
311
Table 1
Preferences and WTP for hysterectomy (WTPH) and conservative surgery (WTPC)
Dichotomous
choicea
WTPC >
WTPH
WTPC
WTPH
WTPC <
WTPH
Cons Hyst
(48.64%)
Hyst Cons
(19.25%)
Not de®nedb
(8.11%)
10
(19%)
0
18
(35%)
5
(26%)
3
(37%)
19
(41%)
14
(74%)
1
(13%)
26
34
Total
0
10
2
v (WTP)
Total
v2 (choice)
12.552
P 0:001
4
(50%)
48
(100%)
19
(100%)
8
(100%)
5
76
Missing
1
(5%)
0
12.302
(0.001)
0.018
0.001
FisherÕs Exact Test
McNemar Test
a
`` '' means ``strictly preferred to''.
ÔNot de®nedÕ responses included seven respondents who did not express a preference and one respondent
who chose both treatments.
b
Table 2
Results from the Probit regression model explaining inconsistent responses
Variable
General model
Coecient
Cost
Occupation
Time completion
Ease completion
N
Log likelihood function
McFadden R2
Chi-squared
)0.79
0.0010
)0.0011
0.19
65
)37.02
0.075
6.19
Speci®c model
P-value
0.03
0.30
0.10
0.00
Coecient
)0.69
)
)
0.19
P-value
0.04
)
)
0.00
65
)38.79
0.03
2.64
preferred hysterectomy were willing to pay signi®cantly more to have this
treatment (£2116 compared to £847; t ÿ4:456; P 0:001) and those that
preferred conservative surgery were willing to pay signi®cantly more for this
treatment (£2571 compared to £1711; t 2:74; P 0:001). Although WTP
values are dierent between the entire sample and consistent respondents,
only WTP for hysterectomy for those preferring conservative surgery (as
expected, due to all the inconsistencies occurring in this group) is signi®cantly
dierent between the two groups (£2290 compared to 1711; t ÿ1:89;
P < 0:05).
312
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Table 3
Mean WTP values for all the sample and the sample of consistent respondents
Entire group
Consistent respondents
t-Statistic (P-value)
Prefer hysterectomy
Mean WTP
Hysterectomy (£) (95% CI)
Mean WTP
Conservative (£) (95% CI)
Strength of preference
t-Statistic (P-value)
2100
(1465.6; 2734.3)
1031
(487; 1574.9)
2.03
t ÿ3:217; P 0:005
2116
(1701.8; 2956.9)
847
(388.3; 1435)
2.55
t ÿ4:456; p 0:001
0.77
(P > 0:15)
)0.48
(P > 0:25)
±
Prefer conservative
Mean WTP
Hysterectomy (£) (95% CI)
Mean WTP
Conservative (£) (95% CI)
Strength of preference
t-Statistic (P-value)
2290
(1625.6; 2954.3)
2031
(1492.2; 2569.7)
0.88
t ÿ0:930; P > 0:15
1711
(1129.9; 2334.3)
2571
(1629.1; 3199.4)
1.39
t 2:74; P 0:001
)1.89
(P < 0:05)
0.99
(P > 0:15)
±
*
The values forming the ratio are signi®cantly dierent at the 95% level.
Signi®cantly dierent WTP values between the entire sample and the sample of consistent respondents at
the 95% level.
**
4. Discussion and conclusions
This paper found a number of important results when considering the
application of WTP experiments to assessing the economic value of a commodity. The results from the WTP experiment were dierent to those from
the dichotomous choice question. This was partly explained by cost-based
WTP responses, with 30% of respondents willing to pay less for their preferred alternative. A number of other WTP experiments have found evidence
of cost-based WTP responses. Schkade and Payne (1994), in using verbal
protocol analysis to investigate how people respond to WTP questions, found
that individuals justify their WTP responses by referring to the cost of the
commodity being valued. Donaldson et al. (1997) found evidence of costbased responses in a study looking at dierent methods of testing for cystic
®brosis. They argued that this problem may arise when open-ended questions
are used to elicit WTP values. The study presented here found evidence of
cost-based responses when using the PC.
Baron and Maxwell (1996), in a study looking at the value of public goods,
provided respondents with dierent types of costing information. They
conclude that future WTP studies should avoid providing respondents with
information on costs. The pilot study reported here indicated that even when
individuals are not provided with information on costs, they still provide
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
313
cost-based responses. Following this, the main study reported here directly
attempted to avoid cost-based responses by including a statement in the
questionnaire explaining that cost is not the same as value. Despite this, the
experiment still found evidence of cost-based responses. Further, such an
approach may have the added disadvantage that highlighting the cost-value
distinction may alert individuals to the concept of cost.
Questions are raised concerning the eect of cost-based WTP on the application of WTP experiments within a cost±bene®t analysis (CBA) framework. If we assume that inconsistent respondents have not understood the
task and hence have not answered in terms of their maximum WTP, their
responses do not re¯ect their maximum value but rather the cost of treatment. If cost is being used as a marker for expressing WTP then WTP for low
cost options may be underestimated and WTP for high cost options may be
overestimated. Inecient policy conclusions may therefore result.
Given that cost-based responses will lead to inaccurate estimates of welfare
changes, how may such responses be avoided? To date it has been shown that
both the open-ended and PC WTP techniques are vulnerable to such responses. Would the referendum style WTP approach also be subject to such
biases? 5 This method was proposed by the US National Oceanographic and
Atmospheric Administration (NOAA) Panel on contingent valuation as the
method of choice in WTP studies of the environment (Arrow, Solow, Portney, Leamer, Radner & Schuman, 1993). 6 This recommendation was based
on the premise that this technique would not be subject to biases, though this
argument has since been challenged (Diamond & Hausman, 1994). Costbased responses may still result from the referendum approach. For example,
if respondents think the commodity being valued costs less than the bid
amount they are presented with, they may be more likely to respond negatively to the bid amount presented to them. Similarly, if they think the
commodity being valued costs more than the bid amount they are presented
with, they may be more likely to respond positively to the bid amount presented to them. This behaviour would be consistent with the Ôfair priceÕ explanation for WTP responses, i.e. people like to pay what they believe it
5
Using this approach, individuals are asked whether they would pay a speci®ed amount for a given
commodity, with possible responses usually being ÔyesÕ or ÔnoÕ. The bid amount is varied across
respondents and the only information obtained from each individual respondent is whether her maximum
WTP is above or below the bid oered (Hanemann, 1984).
6
The NOAA panel was a group of ÔexpertsÕ who oered guidelines on the use of WTP for
environmental resource damage assessments.
314
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
costs. They do not want to be exploited by paying more than the commodity
costs, nor to exploit by paying less than the commodity costs. There is evidence of this Ôfair-priceÕ behaviour in the provision of both private and public
goods (Kahneman, Knetsch & Thaler, 1986; Winer, 1986; Green, Kahneman
& Kunreuther, 1994).
One way to attempt to overcome cost-based responses is to focus on the
bene®ts from the commodities being valued (Baron & Maxwell, 1996). The
marginal WTP approach may be used to do this (Donaldson et al., 1997;
Donaldson, Hundley & Mapp, 1998). Here individuals are ®rst asked what
option or treatment they prefer. They are then asked their maximum WTP to
have their preferred option over their less preferred. A second alternative is
the use of conjoint analysis (CA) discrete choice models to estimate WTP.
Using CA respondents are presented with numerous discrete choices, which
dier in terms of the attributes of the service, and, for each choice, they are
asked to choose their most preferred. By estimating a dierence model, and
including cost as an attribute, marginal WTP values for the change from one
treatment to another can be obtained (Ryan & Hughes, 1997). Future work
should explore whether cost-based responses emerge when these instruments
are employed.
Even if the potential problem of cost-based responding in WTP experiments is successfully overcome, research has indicated a number of other
problems with using this technique to elicit the economic value of a good.
These include the insensitivity of WTP estimates to factors that might be
expected to in¯uence such responses (Kahneman, 1986; Kahneman &
Knetsch, 1992; Diamond & Hausman, 1994; Irwin & Spira, 1997); the sensitivity of WTP estimates to factors which are theoretically irrelevant to the
commodity being valued (Bishop, Heberlein & Kealy, 1983; Sellar, Chavas &
Stoll, 1985; Cummings et al., 1986; Peterson, Driver & Gregory, 1988;
Johnson, Bregenzer & Shelby, 1990; Samples & Holier, 1990; McFadden &
Leonard, 1992; Boyle, Johnson, McCollum, Desvousges, Dunford & Hudson, 1993; Irwin, Slovick, Lichenstein & Mclelland, 1993; Kealy & Turner,
1993; McFadden, 1994; Ready, Buzby & Hu, 1996; Kramer & Mercer, 1997);
the presence of strategic behaviour (Bohm, 1972; Cummings et al., 1986;
Mitchell & Carson, 1989; Posavac, 1998); and the extent to which values exist
for unfamiliar goods (Fischo, 1991; Payne & Bettman, 1993).
In conclusion, this paper developed a method to test for the consistency of
responses in WTP experiments ± if commodity A is preferred to B, then individuals should be willing to pay more for A than B. Thirty percent of respondents fail to provide consistent responses. One reason for such
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
315
inconsistencies was cost-based responses. These ®ndings are consistent with
other studies in the WTP literature. The results are important when considering the application of WTP experiments both in health care, and elsewhere. Whilst economists argue the case for the use of WTP on the grounds
that the instrument provides a cardinal measure of preferences, the evidence
presented here suggests that the technique may not even provide an ordinal
measure of preferences. Possible solutions to avoid cost-based WTP responses include the marginal WTP approach and CA. Future work should
explore the consistency of these approaches, as well as the validity of WTP
experiments more generally.
Acknowledgements
The authors would like to thank Emma McIntosh and two anonymous
referees for comments on this paper. Financial support from the Medical
Research Council (MRC), the Chief Scientist Oce of the Scottish Oce
Department of Health (SODOH), and the government of Navarre are acknowledged. The views expressed in the paper are those of the authors.
References
Arrow, K., Solow, R., Portney, P., Leamer, E., Radner, R., & Schuman, H. (1993). Report of the NOAA
panel of contingent valuation, Washington, DC. Federal Register, 10, 4601±4614.
Baron, J., & Maxwell, N. (1996). Cost of public goods aects willingness to pay for them. Journal of
Behavioural Decision Making, 9, 173±183.
Bland, M. (1995). An introduction to medical statistics. Oxford: Oxford University Press.
Bishop, R., Heberlein, T., & Kealy, M. (1983). Contingent valuation of environmental assets:
Comparisons with a simulated market. Natural Resources Journal, 23, 619±633.
Bohm, P. (1972). Estimating demand for public goods: An experiment. European Economic Review, 3,
111±130.
Boyle, K., Johnson, F., McCollum, D., Desvousges, W., Dunford, R., & Hudson, S. (1993). Valuing public
goods: Discrete versus continuous contingent-valuation responses. Department of Resource Economics,
Working Paper, University of Maine.
Cameron, I. M., Mollison, J., Pinion, S. B., Atherton-Naji, A., Buckingham, K., & Torgerson, D. (1996).
A cost comparison of hysterectomy and hysteroscopic surgery for the treatment of menorrhagia.
European Journal of Obstetrics and Gynaecology and Reproductive Biology, 70, 87±92.
Carson, R., Wright, J., Alberini, A., Carson, N., & Flores, N. (1994). A bibliography of contingent
valuation studies and papers. La Jolla, CA: Natural Resource Damage Assessment.
Cummings, R. G., Brookshire, D. S., & Schulze, W. D. (1986). Valuing environmental goods: A state of the
arts assessment of the contingent method. Totowa: Rowman and Allanheld.
316
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
Diamond, P., & Hausman, J. (1994). Contingent valuation: Is some number better than no number?
Journal of Economic Perspectives, 8, 45±64.
Diener, A., O'Brien, B., & Gafni, A. (1998). Health care contingent valuation studies: A review and
classi®cation of the literature. Health Economics, 7, 313±326.
Donaldson, C., Shackley, P., Abdalla, M., & Miedzybrodzka, Z. (1995). Willingness to pay for antenatal
carrier screening for cystic ®brosis. Health Economics, 4, 439±452.
Donaldson, C., Shackley, P., & Abdalla, M. (1997). Using willingness to pay to value close substitutes:
Carrier screening for cystic ®brosis revisited. Health Economics, 6, 145±159.
Donaldson, C., Hundley, V., & Mapp, T. (1998). Willingness to pay: A method for measuring preferences
for maternity care? Birth, 25, 32±39.
Fischo, B. (1991). Value elicitation: Is there anything in there? Americam Psychologist, 46, 835±847.
Green, D., Kahneman, D., & Kunreuther, H. (1994). How the scope and method of public funding aects
willingness to pay for public goods. Public Opinion Quarterly, 58, 49±67.
Hanemann, W. (1984). Welfare evaluations in contingent valuation experiments with discrete responses:
Reply. American Journal of Agricultural Economics, 69, 332±341.
Irwin, J. R., Slovic, P., Lichenstein, S., & McClelland, G. H. (1993). Preference reversals and
measurements of environmental values. Journal of Risk and Uncertainty, 6, 5±18.
Irwin, J. R., & Spira, J. S. (1997). Anomalies in the values for consumer goods with environmental
attributes. Journal of Consumer Psychology, 6, 339±363.
Johannesson, M., Jonsson, B., & Borgquist, B. (1991). Willingness to pay for antihypertensive therapy.
Results of a Swedish pilot study. Journal of Health Economics, 10, 461±474.
Johannesson, M., Johansson, P.-O., Kristrom, B., & Gerdtham, U.-G. (1993). Willingness to pay for
antihypertensive therapy ± further results. Journal of Health Economics, 12, 95±108.
Johnson, R. L., Bregenzer, N. S., & Shelby, B. (1990). Contingent valuation question formats:
Dichotomous choice versus open-ended responses. In R. L. Johnson, & G. V. Johnson, Economic
valuation of natural resources: Issues, theory and applications (pp. 193±204). Boulder, CO: Westview
Press.
Kahneman, D. (1986). Comments on the contingent valuation method. In R. G. Cummings, D. S.
Brookshire, & W. D. Schulze, Valuing environmental goods: An assessment of the contingent valuation
method. Savage, MD: Rowman and Little®eld.
Kahneman, D., Knetsch, J., & Thaler, R. (1986). Fairness as a constraint on pro®t seeking. Entitlements in
the market. American Economic Review, 76, 728±741.
Kahneman, D., & Knetsch, J. L. (1992). Valuing public goods: The purchase of moral satisfaction. Journal
of Environmental and Management, 22, 57±70.
Kealy, M., & Turner, R. (1993). A test of equality of closed-ended and open-ended contingent valuations.
American Journal of Agricultural Economics, 75, 321±331.
Kramer, R. A., & Mercer, D. E. (1997). Valuing a global environmental good: US residents' willingness to
pay to protect tropical rain forests. Land Economics, 73, 196±210.
McFadden, D. (1994). Contingent valuation and social choice. American Journal of Agricultural
Economics, 76, 689±708.
McFadden, D., & Leonard, G. (1992). Issues in the contingent valuation of environmental goods:
Methodologies for data collection and analysis. Berkeley, CA: University of California.
Mitchell, R. C., & Carson, R. T. (1989). Using surveys to value public goods. Washington, DC: Resources
for the Future.
O'Brien, B., & Viramontes, J. (1994). A valid and reliable measure of health state preference? Medical
Decision Making, 14, 289±297.
Payne, J. W., & Bettman, J. R. (1993). The adaptive decision maker. New York: Cambridge University
Press.
Peterson, G., Driver, B., & Gregory, R. (1988). Amenity resource valuation: Integrating economics with
other disciplines. State College, PA: Venture publishing.
M. Ryan, F. San Miguel / Journal of Economic Psychology 21 (2000) 305±317
317
Posavac, S. S. (1998). Strategic overbidding in contingent valuation: stated economic value of public goods
varies according to consumersÕ expectations of funding source. Journal of Economic Psychology, 19,
205±214.
Pinion, S. B., Parkin, D. E., Abramovich, D. R., Naki, A., Alexander, D., Russell, I. T., & Kitchener, H.
C. (1994). Randomised trial of hysterectomy, endometrial laser ablation and transcervical endometrial
resection for dysfunctional uterine bleeding. British Medical Journal, 309, 979±983.
Ready, R. C., Buzby, J. C., & Hu, D. (1996). Dierences between continuous and discrete contingent value
estimates. Land Economics, 72, 397±411.
Ryan, M., & Hughes, J. (1997). Using conjoint analysis to assess womenÕs preferences for miscarriage
management. Health Economics, 6, 261±273.
Ryan, M., Ratclie, J., & Tucker, J. (1997). Using willingness to pay to value alternative models of
antenatal care. Social Science and Medicine, 44, 371±380.
Samples, K. C., & Holier, J. R. (1990). Contingent valuation of wildlife resources in the presence of
substitutes and compliments. In R. L. Johnston, & G. V. Johnston, Economic valuation of natural
resources: Issues, theory, and applications. Boulder, CO: Westview Press.
Schkade, D., & Payne, J. (1994). How people respond to contingent valuation questions: a verbal protocol
analysis of willingness to pay for an environmental regulation. Journal of Environmental Economics and
Management, 26, 88±109.
Sellar, C., Chavas, J. P., & Stoll, J. R. (1985). Validation of empirical measures of welfare change: A
comparison of nonmarket techniques. Land Economics, 61, 156±175.
Sen, A. (1993). Internal consistency of choice. Econometrica, 61, 495±521.
Winer, R. (1986). A reference price model of brand choice for frequently purchased products. Journal of
Consumer Research, 13, 250±256.