Pilot testing the assessment instrument

2.7 Pilot testing the assessment instrument

Many authors or researchers of literacy assessment emphasize the importance of pretesting or pilot testing the assessment instrument. Wagner 2004:35 noted that ―a ‗testing the test‘ development process is essential to enhance the reliability, validity, utility, and feasibility of the measures themselves before broad assessment is undertaken. ‖ Comings 2009:3 described the development process of the literacy assessment instrument as three phases: design phase, tryout phase, and pilot phase. As he noted, the goal of a tryout with a small sample of population is to find out whether the testing instrument is too easy or too difficult and whether or not it is measuring differences between test takers with different levels of literacy skill ibid.:8. Similarly, Walter and Davis, the Eritrea National Reading Survey team, pointed out that the results of the pilot test were used to revise the instruments used for the final reading survey Ministry of Education of the State of Eritrea, 2005:17. The first sample draft English version of the literacy assessment for the Eastern Tamang adult learners also needs to go through the tryout or pilot testing phase. It would also be sent to several MT-base adult literacy programs such as Magar and Tharu to be rendered into different mother tongues and to be tested in one or more of their literacy classes. More detail of pilot testing will be discussed in Chapter 5 which deals with the methodology of literacy assessment. 68 3 CONSIDERATIONS FOR DEVELOPING A LITERACY ASSESSMENT INSTRUMENT IN NEPAL As literacy is a means to broader ends for individuals and communities, a literacy program should be carefully planned in regards to the essential factors that affect the status and welfare of those individuals and communities Gray, 1956:252. Likewise, an assessment program is a critical means to identify and serve significant information needs of diverse stakeholders involved in a literacy program. Thus, the process of developing a literacy assessment instrument should also be carefully planned to meet these needs. However, there is little literature giving clear guidelines on the assessment of adult literacy in non-Indo-European languages. This chapter aims: 1 to derive general guidelines for developing an effective and adaptable assessment procedure for the Eastern Tamang learners from the existing literature on the subject of evaluation of literacy in both formal and informal educational programs in industrialized countries; 2 to set a potential Eastern Tamang operational definition of literacy for the purpose of assessment; and 3 to select specific literacy skills and test components and to pre-set or posit minimum skill levels as criteria for success for Eastern Tamang adult learners in the ETLT literacy program.

3.1 General guidelines for a planning literacy assessment