Use and assessment of indicators

142 R.D.B. Lefroy et al. Agriculture, Ecosystems and Environment 81 2000 137–146 Table 5 Acceptability indicators and threshold ratings, with farmer rankings and comments Indicator Qualitative ratings Farmer importance ranking Farmer comments Very Medium Not Tenurial status Full ownership 20 Long term user rights No official land title Access to extension services Full technical support 12 7 1 Limited technical support No support Access to primary schools 1 km 8 12 1–3 km 3 km Access to health centre 3 km 7 12 1 3–5 km 5 km Access to agricultural inputs 5 km 5 18 7 5–10 km 10 km Subsidy for conservation practices 50 15 5 10–20 of total cost 25–50 25 No subsidy Training in conservation practices Once in 3 years 18 2 School for young farmers Once in 5 years None available Village road links to major roads Full access 1 17 2 Limited access No access by the 20 Vietnamese farmer-collaborators when an early version of the DSS was first used to assess the sustainability of their farming systems. The farmer rankings indicate their assessment of the importance and relevance of the indicators and the qualitative ratings with respect to sustainability, while their com- ments indicate alternative or additional indicators and alterations to threshold ratings.

4. Use and assessment of indicators

4.1. Choosing appropriate indicators When data were collected from the field, the use of complex measurements was, as far as possible, avoided so extension workers can use the methodol- ogy in the future. This had particular bearing on the biophysical data. The collection and analysis of soil and plant samples and the accurate measurement of yield, nutrient inputs, erosion and runoff would have enabled more direct comparisons between farmers in the same and in different areas, but such analyses are beyond the resources of most groups involved in im- plementing SLM strategies. Comparisons of the more practical descriptive analyses collected in these stud- ies are more difficult and less precise. This problem highlights the need to develop and use reasonably accurate surrogate indicators of im- portant characteristics of farming systems, such as soil fertility, nutrient balance, and erosion, runoff and leaching Fairhurst et al., 1998. Such surrogate indi- cators would enable extension workers and farmers to develop and implement improved land management R.D.B. Lefroy et al. Agriculture, Ecosystems and Environment 81 2000 137–146 143 strategies without recourse to laboratory analyses, and enable them to assess the relative sustainability of different farming systems. On reflection, the inclusion of more specific measurements in these case studies may have been a useful step to develop improved surrogate measures, using correlation analyses, that could be used as indicators of SLM. Gomez et al. 1996 used correlation analysis to choose appropriate indicators from a suite of biophys- ical factors. Through this process, they identified in- dicators that represented what they considered to be major components of sustainability: indicators that ex- hibited sufficient variance to indicate sensitivity to changes in management, were highly correlated with associated indicators, poorly correlated with indica- tors of other components of sustainability, and were relatively cheap and easy to measure. Another problem with data of this nature is that it is highly inter-related. In many cases, the indicators identified in this study are pre-indicators or evalua- tion factors, which need to be integrated to develop appropriate indicators. For instance, information on inputs of fertiliser and organic residues are not good indicators, but when used together, and in conjunction with information on residue management and yield, they can result in an estimate of nutrient balance, which can be used as an indicator. Similarly, while farm income is important, it must be interpreted along with information on subsistence food production, debt servicing, and other expenditure. Assessment of the socio-economic factors from the site in Thailand highlights a number of these complex interactions and trends. By identifying these factors and characteristics, and by establishing and under- standing their interactions and trends, it is hoped that single evaluation factors can be combined and repre- sented by an indicator that reflects more accurately the sustainability of a particular land-use in a defined locality. Threshold levels of these selected integrated indicators can then be developed to cover a range of systems. An example of the problems that arise with sin- gle factor analyses is the assessment of political involvement by villagers. For instance, within the macro-political framework, the villagers were as- sessed to have a low degree of political participation and an unclear legal status, due to their status as an ethnic minority group. Their presence as farmers on marginal land claimed by the state is tolerated, al- though their freedom of movement is sometimes lim- ited. Due to the low degree of political participation, the upland minorities are supported by Thai NGOs, which are allowed to operate freely and provide de- velopment assistance. NGOs and other groups, such as academics, have become influential advocates for the interests of minority groups, and, in some cases, have lobbied successfully to improve their legal status and their access to resources and infrastructure. Thus, although the answers to some questions indicate their direct political participation is limited, their situation, as revealed through more integrated PRA techniques, is not as serious as if they had no influential advo- cates. Developing an indicator for this situation is very complex, but especially when this is attempted with a single factor as the sole measure. Similarly, the issues of labour, farm income, and debt are very complex and inter-linked. Until the impact of the current economic downturn, there was an acute labour shortage in northern Thailand due to the labour intensive nature of upland farming and the opportunities for off-farm employment in one of the country’s booming urban areas or abroad. As a result of off-farm employment, farm incomes increased in recent years, with mixed impact on farming systems. While some farmers invested part of their increased earnings in purchasing modern inputs for their farms or improving their farming systems, others, quite un- derstandably, acquired consumer goods or built a new existence in their new, mostly urban surroundings. As such, assessment of off-farm income is only a component of the direct impact on farming systems. Evaluation factors like labour, income, and debt need to be translated into more meaningful, integrated or composite indicators. 4.2. Development of a prototype decision support system to assess sustainability A prototype DSS was developed, to assess the sta- tus of each of the five pillars, using the responses to 26 questions based on the indicators listed in Tables 1–5. The answers to each question are characterised as hav- ing a weighted impact on the particular pillar to which they are associated in terms of four discrete categories: i meeting the requirements of sustainability, ii be- ing marginally above the threshold for sustainability, 144 R.D.B. Lefroy et al. Agriculture, Ecosystems and Environment 81 2000 137–146 iii being marginally below the threshold, or iv not meeting sustainability. Summation of the various in- dicators translates to a score between 0 and 1 for each pillar. The overall sustainability of the system is then assessed on the status of the five pillars, and most par- ticularly on the status of the least sustainable pillar. The aim of this exercise was to assess the sustain- ability of smallholder farmers in sloping uplands of SE Asia. As was expected, highly site-specific indi- cators were developed. In addition, the relevance of some of these indicators to these specific sites was limited due to the effort to maintain simplicity. While many of these indicators can be made more general, it is unlikely that a truly generic approach to SLM assessment will eventuate. Validation of indicators of SLM and of the DSS is not easy due to the complex nature of sustainability and the necessity for medium to long-term evaluation; they cannot be verified with a precise measure of sus- tainability. Feedback from the farmers and researchers helped to increase confidence in the process. In a sub- sequent study, the prototype was adapted for use in the middle hills region of Nepal with input from farmers, Table 6 Results of sustainability assessments using two related versions of the DSS in a sloping lands of Vietnam V and Indonesia I, and b the middle hills region of Nepal N a ,b Pillars Assessment categories Meets sustainability Marginally above threshold Marginally below threshold Not sustainable Productivity V2, V3, V5 V1, V4 I1, I2 N3, N4, N5, N7 N1, N2, N6, N9 N10 N8 Security V1, V3, V4 V2, V5 I1, I2 N1, N2, N3, N4, N5, N6, N7, N8, N9 N10 Protection V1, V2, V4, V5 V3 I1, I2 N1, N2, N3, N4, N5, N6, N7, N8, N9, N10 Economic viability V1 V2, V3, V4, V5 I1, I2 N1, N4, N5, N6, N7 N2, N3 N8, N9, N10 Social acceptability V1, V2, V3, V4, V5 I1, I2 N1, N2, N3, N4, N5, N6, N7, N8, N9, N10 a Bold symbols indicate the farms assessed as sustainable for all five pillars of sustainability. b Source: data for Nepal from Rais et al. 1998b. extension officers and researchers Rais et al., 1998b. The same basic structure was used with changes to the questions to suit the agroecological and socio-cultural systems. One indicator was added, to take into ac- count the potential damage from hail and windstorms, the answers andor thresholds of 18 questions were changed to suite the different farming systems and environment, and eight essentially remained the same. In this Nepali study, the use of the DSS structure as a process to quickly adapt a system of SLM evaluation was deemed successful. Of course, the accuracy with which sustainability was assessed could not be veri- fied. These assessments indicated only three out of 10 farms as sustainable, with the majority of unsustain- able farms being considered so in terms of economic viability, and no farms considered unsustainable in terms of social acceptability Table 6. These results compare with those of five farmers in the Vietnam study and two in the Indonesia study. Although none of these seven farming systems were assessed as meeting the requirements of sustainability for all the five pillars, all were at least marginally above the threshold for sustainability. In contrast, some of the R.D.B. Lefroy et al. Agriculture, Ecosystems and Environment 81 2000 137–146 145 Nepali farms were assessed as being marginally be- low the threshold, or not meeting the requirements for sustainability.

5. Conclusions