Questionnaire design Directory UMM :Data Elmu:jurnal:E:Ecological Economics:Vol32.Issue2.Feb2000:

been attempting to increase the carrying capacity of their land by a variety of methods, including the clearing of trees and the introduction of non- native grass species. Initially these developments were limited to patches of more fertile soils. The region now has one of the highest clearing rates in Australia, with between 4 and 8 of many broad country types being cleared between 1992 and 1995 McCosker and Cox, 1996. Landholders must gain permission to clear trees from the Queensland Government through the Department of Natural Resources. In issuing the permits for broadscale tree clearing, the State Government policy calls for a balance between the benefits of increased productivity most of which accrue directly to the landholders against the environmental costs of diminished vegetation cover which are more broadly spread across the regional and national communities. The estima- tion of these environmental costs is the focus of the CM application described below. The Queensland Government has recently been revising its tree clearing policies, with the result that some vegetation communities are now pro- tected. Other vegetation communities can be cleared to 20 of their original extent on individ- ual properties. The CM exercise is aimed at as- sessing the environmental costs associated with alternate guidelines. The use of labels reflecting tree retention levels is thus the most policy relevant way of framing choice options. In the labelled version of the questionnaire employed in this study, respondents were asked to choose whether they prefer the current retention level of 20, increased retention of 30, or a further increase to 50. 5 Note that these labels communicate information regarding changes in protected vegetation area.

6. Questionnaire design

The questionnaire design phase involved exten- sive background research and two rounds of focus groups with potential respondents. 6 The first round of focus groups focussed on the identifica- tion of key decision parameters from the respon- dent’s perspective. This information played an important role in the selection of attributes. Table Table 1 Attributes, levels and corresponding variables Attribute Levels in generic Levels in alt- spec. model model Option A: 0 Levy on income Option A: 0 tax base base Option B: 20, Options B and C: 40, 60, 80 20, 60, 100, 140 Option C: 80, 100, 120, 140 Option A: 0 Option A: 0 Income lost to re- gion in million Options B and C: Option B: 5, 7, 5, 10, 15 9 Option C: 11, 13, 15 Option A:0 Jobs lost in region Option A:0 Options B and C: Option B: 10, 14, 18, 21, 24 10, 15, 20, 30, 40 Option C: 26, 30, 34, 37, 40 Number of endan- Option A:18 Option A:18 gered species lost to region Option B: 10, Option B and C: 4, 8, 12, 16 12, 14, 16 Option C: 4, 6, 8, 10. Reduction in popu- Option A: 80 Option A:80 lation size of non-threatened species Option B: 60, Option B and C: 30, 45, 60, 65, 70, 75 75 Option C: 30, 35, 40, 50 6 The first set of focus groups involved two in Brisbane and a further two in Emerald. The second set involved two groups in Brisbane. 5 Focus group research indicated that a fourth option per- taining to less than 20 retention was neither policy relevant nor considered a viable option by respondents. 1 lists the attributes chosen. 7 Note that environmental losses receive equal attention in the choice sets and elsewhere in the questionnaire to losses in jobs and regional in- come. Inclusion of the latter, ‘developmental’ at- tributes, is consistent with Portney’s observation Portney, 1994 that people can have nonuse values for economic and social factors. The balanced environment – economy approach to information presentation addresses Blamey’s 1996, p. 128 concern that it is often futile to omit or downplay references to such outcomes ‘in the hope that respondents will not bring their own perceptions of such factors in as external variables’. Citizens have general stances on environmental issues that they bring to the valuation situation, and implicit in these stances is a consideration of the relative importance of environmental factors and developmental factors. Omitting or downplaying the development side of the story not only leads some respondents to perceive the questionnaire to be biased, but also results in the elicitation of a blurred construct. The objective of environmental valuation is to estimate environmental consumer surplus. An alternative to seeking surplus validity is to seek a form of predictive validity pertaining to the maximum amount of money citizens are prepared to commit themselves to paying at a referendum electoral validity. Developmental considerations and payment vehicle protests are invalid from the former perspective but valid from the latter. Unbalanced information presentation results in a blurred compromise between these two objectives. In this paper, we have attempted to maximise predictive WTP validity and to interpret WTP for environmental attributes within this light. To the extent that losses in regional income and employment are expected to be short-lived, lower impacts, or zero impacts, can be substituted into the model as an approximation when estimating WTP and market share. Other stated preference studies that have included developmental benefits include Lockwood et al. 1994, Morrison et al. 1999. The second round of focus groups focussed on the refinement of draft questionnaires. Some of the more important issues that emerged concerned the clarity of information; selection of photographic stimuli; cognitive burden particularly as it relates to the number of choice sets; perceived bias of information presented; strategies for choice; and plausibility of attribute-combinations. Particular attention was also given to whether individuals interpreted information and questions in the way intended by the researchers. Different ways of introducing and explaining the choice modelling task were also explored. Both generic and alternative-specific choice set configurations were trialed, and in some cases, in the same group. 8 Whilst some participants thought alternative-specific labels were a good idea, others were not so sure: I think this puts a whole different spin on things....I think it makes the decision more realistic. E1 If you have that [label] on every single one, then people might just look at that… They might just go ‘I’ll pick the one with the most trees’. B2 A common tendency was to select the option with the label that most closely coincided with one’s environmental attitudes, and to see if significant reasons existed not to choose this alternative. The following statements illustrate how some individu- als used the labels to help structure their evaluations: Straight away I thought yeah I do want to do something to increase it [the minimum permissi- ble tree retention] and I don’t mind paying some money. B3 8 The particular group to which the verbatim quotes listed below correspond is indicated at the end of the citation for each quote, using the abbreviations B1 – 4 and E1 or E2. B2 thus refers to the second focus group conducted in Brisbane. 7 Inclusion of an attribute pertaining to the percentage of tree retention did not appear to be necessary: participants appeared to be more focussed on final outcomes such as the number or percentage of species and jobs lost or saved. This suggests that the omission of an ‘area’ attribute is unlikely to result in significant omitted variable bias. More likely, inclu- sion of the above-mentioned policy names in the second version of the questionnaire may prompt considerations that would not otherwise occur. Putting the top line [labels] up there makes it a bit easier. B3 The final questionnaires were presented in the form of a small booklet with a colour cover, and a colour insert containing an attribute glos- sary for use when completing the choice sets. A map on the cover indicated the location of the Desert Uplands in Queensland, and proximity to nearby towns. A graphic artist finalised the pre- sentation of the questionnaire and pamphlet. The levels assigned to the attributes listed in Table 1 were chosen such that the resultant at- tribute-space encompassed the vast majority of policy-relevant tree clearing options. Information regarding the ecological effects of different tree clearing options in the Desert Uplands, and consequent implications for humans, is ex- tremely limited. Information that could be gained regarding likely outcomes for jobs, re- gional income and threatened and non-threat- ened species was obtained from a variety of sources summarised in Rolfe et al., 1997 and in consultation with experts. The high level of uncertainty regarding attributes such as impact on endangered species meant that the range of levels chosen was wider than one would expect to be the case with most policy options. Because most respondents would expect greater tree clearing to involve worse environ- mental outcomes and better economic outcomes, attribute values in the labelled treatment were selected from alternative-specific sets of values. In other words, different policy labels were asso- ciated with different sets of outcomes. Table 1 lists the attribute levels applied to each option in each treatment and Fig. 1 illustrates the main differences between the two treatments. 9 The ca- pacity to incorporate associations between at- tribute values and labels within the design of the experiment can be considered an advantage of the alternative-specific approach to choice set design. Implausible combinations of attributes and labels are minimised as a result. 10 To ensure that the attributes varied indepen- dently of one another, such that their individual effect on respondents’ preferences can be iso- lated, an orthogonal experimental design was used to assign attribute levels to alternatives. Fractional factorial designs were used to reduce the number of alternatives to a manageable level. Choice sets were constructed in such a way that orthogonality both between and within alternatives was ensured. To reduce implausibil- ity problems whilst at the same time increasing the balance between environmental and eco- nomic variables, a correlation between jobs lost and income lost was introduced by creating a composite 8 level attribute. Sixty-four choice sets were allocated to eight blocks of eight choice sets in each of the two versions, produc- ing a total of 16 versions of the questionnaire. An eight-level, orthogonal blocking variable was included as part of the experimental design, and used to generate the eight different blocks or versions of the survey. The purpose of the blocking factor is to insure that the eight blocks feature a balanced distribution of levels across all attributes, which ordinarily cannot be achieved using random assignment. In both versions of the questionnaire, state- ments were included in the scenario with the purpose of further diffusing perceptions of im- plausible attribute combinations that may give rise to problematic response strategies. Respon- dents were told to ‘consider carefully the impli- cations of each tree-clearing option by looking at the numbers in the table. To keep matters simple, we do not describe how each option would work. Some implications which may seem a little odd are in fact quite possible…You will find some questions easier than others.’ 10 Whether or not alternative-specific attribute levels are required clearly depends to a large extent on the nature of the information communicated by the labels. For example, labels based on biogeographic differences such as ‘The Desert Up- lands’ and ‘The Brigalow’ may not require alternative-specific attribute levels if respondents do not have strong a priori expectations regarding the relative magnitude of attribute lev- els for these regions. 9 Every attempt was made to keep the two main versions of the questionnaire as similar as possible. Minor wording differ- ences were, however, required in two paragraphs prior to the choice sets in order to bring the scenarios in line with them. Fig. 1. a An example of a generic choice set. b An example of an alternative-specific choice set.

7. Survey logistics