Designing a set of web-based multiple-choice reading test for the grade XII students of senior high school in Kecamatan Cibinong.

i  

DESIGNING A SET OF WEB-BASED MULTIPLE-CHOICE READING TEST FOR THE GRADE XII STUDENTS OF SENIOR HIGH SCHOOL

IN KECAMATAN CIBINONG

A THESIS

Presented as Partial Fulfillment of the Requirements to Obtain the Sarjana Pendidikan Degree

in English Language Education

By

Antonia Fitri Lestari Student Number: 051214147

ENGLISH LANGUAGE EDUCATION STUDY PROGRAM DEPARTMENT OF LANGUAGE AND ARTS EDUCATION FACULTY OF TEACHERS TRAINING AND EDUCATION

SANATA DHARMA UNIVERSITY YOGYAKARTA

ii  

iii  

iv  

v  

 

what

live

I dedicat e t o my bel t o al l l ear and t o my

“If w

t are

es,

th

tha

so

ed t his t hes l oved f amil

rners and t y best f rien

we m

goin

hen w

at goa

omeh

sis f or my d ly, t eachers, nds. vi

make

ng to

work

al,

we

how

dearest one

up

o

mak

hard

e nev

we w

e,

our m

ke

of

d tow

ver lo

win o

~Rona

mind

f

our

ward

ose

-out.”

ald Reagan~

vii  

ACKNOWLEDGEMENTS

My endless gratitude goes to The Almighty God for His love, blessing, and opportunities given to me since I started writing this thesis. When I was happy and sad, He always accompanies me. As I believe, God designs my life and keeps me saved in His protection.

I would like to express my greatest appreciation to my major sponsor, Gregorius Punto Aji, S.Pd., M.Hum., for always guiding, encouraging, and supporting me patiently in finishing my thesis through his suggestion, guidance, feedback, and criticism. I am also faithfully thankful to all lecturers of English Language Education Study Program who have taught and guided me in passing my study time in PBI and all staffs in PBI who help me in administrative needs.

I am sincerely thankful to the principal of SMA Mardi Waluya Cibinong and SMA Eka Wijaya Cibinong, Aloysius R. Subandriyo, S.Pd. and Hari Hamzah, S.Kom. who have permitted me in conducting my thesis. Then, I would like to express my gratitude for all English teachers in SMA Mardi Waluya Cibinong and SMA Eka Wijaya Cibinong, Evarista Retno Baskoro Wulan, S.S., Malvin Yusdian, S.Pd., M.Pd., Dendy Wibowo, S.S., Bambang Rosandy, S.Pd., and Scolastica Wedhowerti, S.Pd., M.Hum. for their contribution as participants and their support for the implementation of the study. Then, I would like to thank for all students of SMA Mardi Waluya Cibinong, SMA Eka Wijaya Cibinong, and SMAN 1 Cibinong who have participated in this study.

viii  

My greatest gratitude goes to my family. I express my grateful to my parents, Lucia Mujiyanti, S.Pd. and Antonius Sartana, S.Pd, and my grandparents Mbah Minarjo, Mbah Paikem, and Mbah Fatimah (†) for their greatest patient, support, and prayers. I apologize for not being on schedule in finishing my study. Then, I also thank my sisters, Agustina Fitriana, Sesilia Windy Gloriana, and Clarisa Primaviolinita, for giving me a lot of affections, for making me laugh, and for giving me support in every ways.

From the bottom of my heart, I would like to express my gratitude to my best classmate friends RA. Triastuti Dian K., Yohanna Kurnita P., Dionisia Novianti, Galuh Dwi Ajeng, Eska Dwi Palupi, Steffani Ratna K., Nancy Paula H., Delfina Dae, Pascalia Yovita, Taufik H. and all of PBI ’05 students for their cherished friendship. Special thanks are also addressed to my cyber tutor who helps me in accomplishing this website, Alexander Cahya Kurniawan.

For what is called comradely, I sincerely thank to all my boardinghouse friends, Trias, Astre, Riska, Ana, Puput, Nana, Eni, Ka Achied, Eli, Lucy, Vita and Regina. I thank them for every laugh and happiness in our lovely time.

My special gratitude goes to my dearest one, Bonaventura Trias Y.P. for his never-ending love, support, and motivation. He taught me to overcome my fear and protected me when I was down. God bless you, always!

Finally, I thank everyone in my life whom I cannot mention all their names for the lovely experience that I felt together with you all. God bless you all!

ix  

ABSTRACT

Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan

Cibinong. Yogyakarta: English Language Education Study Program, Sanata Dharma University.

Reading is one of the essential skills for success in all educational contexts because it has function to convey information, to entertain, and to codify. Reading skill may facilitate the learners to master the language. It is because reading integrates the text and the readers’ background knowledge to build meaning. Hence, reading is often implemented and refers to be dominant in a language test. In this study, the writer considered using web-based test in order to create a different way in conducting test. Besides, the web-based test can be accessed by many students without limitation of time and place.

This study is aimed to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The writer intended to answer a problem: What is the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong?

Dealing with the design, the writer applied theories related to theory of Educational Research and Development (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, and Application Software. Moreover, the design was performed step by step according to the R&D cycle combined with Kemp’s model. They are (1) Learners’ Characteristics and Needs, (2) Goal, General Purposes, and Topics, (3) Learning Objectives, (4) Test Content, (5) Activities and Sources, (6) Support Services, (7) Feedback of the Preliminary Designed Test, (8) Revision of Preliminary Designed Test, (9) Feedback from the Implementation, and (10) Final Test Revision.

The appropriate design in the form of web-based multiple-choice reading test is based on data obtained from a research. The writer developed the website address at www.tryoutbahasainggris.co.nr. The writer utilized application software such as Adobe® Flash® and Wondershare QuizCreatorv 3.2 serial. The website includes seven main pages. They are index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html and link.html. The designed test contains of four topics: narrative, explanation, discussion, and review texts. The test content is divided into two groups: reading test and grammatical test. For the activities, the writer implemented three tasks classified into reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Each test consists of 35 questions, 5 options and 100 minutes time constraint. The result of the implementation of designed test showed that the average mean of the whole data was 3.77 fromthe scale 5.0.

In conclusion, the implementation result strengthens that the test was appropriate and acceptable for the Grade XII students of senior high school in Kecamatan Cibinong.

x  

ABSTRAK

Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan

Cibinong. Yogyakarta: Program Studi Pendidikan Bahasa Inggris, Universitas Sanata Dharma.

Membaca merupakan salah satu keahlian yang bagi keberhasilan dalam semua konteks pendidikan karena memiliki fungsi untuk menyampaikan informasi, untuk menghibur, dan untuk menyusun kata-kata. Keterampilan membaca dapat memfasilitasi peserta didik untuk menguasai bahasa. Hal ini dikarenakan membaca menggabungkan teks dengan latar belakang pengetahuan pembaca untuk membangun makna. Oleh karena itu, membaca seringkali dilaksanakan dan menjadi acuan utama dalam sebuah test bahasa. Dalam penelitian ini, penulis mempertimbangkan untuk menggunakan tes berbasis web untuk menciptakan cara yang berbeda dalam melakukan tes. Selain itu, tes berbasis web dapat diakses oleh siswa tanpa batasan waktu dan tempat.

Penelitian ini bertujuan untuk merancang satu set multiple-choice reading tes berbasis web untuk siswa kelas XII SMA di Kecamatan Cibinong. Penulis bermaksud untuk menjawab suatu permasalahan, yaitu: seperti apa rancangan satu set multiple-choice reading tes berbasis web yang sesuai untuk siswa kelas XII SMA di Kecamatan Cibinong.

Berhubungan dengan rancangan tersebut, penulis menggunakan teori-teori yang berkaitan dengan metode Penelitian dan Pengembangan (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, dan Aplikasi Software. Selain itu, desain menggunakan langkah-langkah sesuai dengan siklus R&D dikombinasikan dengan model design Instruksi Kemp. Langkah-langkah yang digunakan yaitu (1) Karakteristik dan kebutuhan siswa, (2) Tujuan umum dan topik, (3) Tujuan pembelajaran, (4) Konten uji, (5) Kegiatan dan sumber, (6) Layanan dukungan, (7) Komentar dari rancangan awal tes, (8) Revisi rancangan awal tes, (9) Komentar dari implementasi tes, dan (10) Revisi dari final tes.

Pembuatan tes didasarkan pada data yang diperoleh dari penelitian. Penulis mengembangkan website dengan alamat www.tryoutbahasainggris.co.nr. menggunakan aplikasi Adobe® Flash ® dan Wondershare QuizCreatorv 3.2. Situs ini mencakup 7 laman, yaitu: index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html dan link.html. Rancangan tes terdiri dari 4 topik yaitu narrative, explanation, discussion, dan review. Isi tes ini dibagi menjadi dua yaitu reading tes dan gramatikal tes. Untuk kegiatan dalam tes, penulis menerapkan tiga jenis tugas yaitu reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Setiap tes terdiri dari 35 pertanyaan, 5 pilihan jawaban dan waktu pengerjaan 100 menit. Hasil pelaksanaan tes menunjukkan bahwa nilai rata-rata dari seluruh data adalah 3,77 dari skala 5,0.

Sebagai kesimpulan, hasil implementasi memperkuat bahwa tes tersebut sudah sesuai dan dapat diterima bagi siswa kelas XII SMA di Kecamatan Cibinong.

xi  

TABLE OF CONTENTS

Page

TITLE PAGE ... i

APPROVAL PAGES ... ii

STATEMENT OF WORK'S ORIGINALITY ... iv

LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI ... v

PAGE OF DEDICATION... vi

ACKNOWLEDGEMENTS ... vii

ABSTRACT ... ix

ABSTRAK ... x

TABLE OF CONTENTS ... xi

LIST OF TABLES ... xv

LIST OF FIGURES ... xvii

CHAPTER I : INTRODUCTION ... 1

A. Research Background... 1

B. Problem Formulation ... 5

C. Problem Limitation ... 6

D. Research Objectives ... 7

E. Research Benefits ... 7

F. Definition of Terms ... 8

CHAPTER II : REVIEW OF RELATED LITERATURE ... 12

A. Theoretical Description ... 12

1. Educational Research and Development ... 12

2. Instructional Design Model ... 16

3. Language Testing ... 20

a. Types of Language Test ... 21

b. Test Qualities ... 23

c. Designing Classroom Language Test ... 25

4. Reading Skill ... 27

xii  

b. Reading Test Types ... 28

c. Reading Assessment ... 30

5. Designing Multiple-choice Items ... 34

6. Web-based Test ... 37

a. The Advantage of Web-based Test ... 37

b. The Role of Computer and Teacher in Web-based Test ... 38

c. Website Evaluation ... 39

7. Application Software ... 41

a. Adobe® Flash® ... 41

b. Wondershare QuizCreator 3.2 ... 41

B. Theoretical Framework ... 43

CHAPTER III : METHODOLOGY ... 47

A. Research Method ... 47

1. Research and Information Collecting ... 48

2. Preliminary Form of Product Development ... 48

3. Preliminary Product Verification ... 49

4. Preliminary Product Revision ... 49

5. Main Field Testing ... 50

6. Final Product Revision ... 50

B. Research Participants ... 51

1. The Participant in Research and Information Colleting Data .... 52

2. The Participant in Preliminary Product Verification ... 52

3. The Participant in Main Field Testing ... 52

C. Research Instruments ... 53

1. The Instrument for Research and Information Colleting Data .. 53

2. The Instrument for Preliminary Product Verification ... 54

3. The Instrument for Main Field Testing ... 54

D. Type of Data and Data Gathering Techniques ... 55

E. Data Analysis Techniques ... 59

1. Analysis Techniques in Research and Information Colleting Data ... 60

xiii  

2. Analysis Techniques in Preliminary Product Verification ... 60

3. Analysis Techniques in Main Field Testing ... 62

F. Research Procedure ... 64

CHAPTER IV : RESEARCH RESULT AND DISCUSSION ... 66

A. The Appropriate Design of a Set of Web-based Multiple-choice Reading Test for Grade XII Students of Senior High School in Kecamatan Cibinong ... 66

1. Learners’ Characteristics and Needs ... 67

2. Goal, General Purposes, and Topics ... 78

3. Learning Objectives ... 80

4. Test Content ... 82

5. Activities and Sources ... 84

6. Support Services ... 86

7. Feedback of Preliminary Designed Test ... 95

8. Revision of Preliminary Designed Test ... 99

9. Feedback from the Implementation ... 109

10.Final Test Revision ... 122

B. The Final Version of the Designed Test ... 125

CHAPTER V : CONCLUSIONS AND SUGGESTIONS ... 126

A. Conclusions ... 126

B. Suggestions ... 128

REFERENCES ... 130

APPENDICES Appendix A: Letters ... 133

1. Letter of Permission to the Headmaster of SMA Mardi Waluya ... 134

2. Letter of Permission to the Headmaster of SMA Eka Wijaya ... 135

3. Letter of Permission from SMA Eka Wijaya ... 136

Appendix B: Instruments ... 137

1. Questions List for Interviewing Teachers and Students ... 138

a. Transcripts of Interview with Teachers ... 140

xiv  

2. Questionnaire for Feedback of Preliminary Designed Test

a. Sample of Filled-questionnaire ... 153

3. Questionnaire for Implementation a. Sample of Filled-questionnaire ... 163

4. Test for the Implementation a. Sample of Filled-test ... 169

b. Results of Open-form Questions from Students’ questionnaire 171 c. Results of Students’ Conducting Tipe B test ... 178

Appendix C: A Set of Web-based Multiple-choice Reading Test for the Grade XII Students of Senior High School in Kecamatan Cibinong ... 181

1. Printed Tipe A test and Explanation ... 182

2. Printed Tipe B test and Explanation ... 196

3. Printed Tipe C test and Explanation ... 210

4. Website Address ... 223

xv  

LIST OF TABLES

Page

Table 3.1: Description Table of the Experts ... 52

Table 3.2: Description Table of the Students ... 53

Table 3.3: Type of Data ... 56

Table 3.4: The Interpretation of the Degree of Agreement ... 61

Table 3.5: The Descriptive Statistics and Transcripts of Respondents’ Opinions... 61

Table 3.6: The Data Collection from Students’ Answer Sheets ... 62

Table 3.7: The Data Collection on Item Facility (IF) ... 63

Table 3.8: The Data Collection on Item Discrimination (ID) ... 63

Table 3.9: The Data Collection on B-Index ... 64

Table 4.1: Summary of Learners’ Characteristics ... 77

Table 4.2: Summary of Learners’ Needs ... 77

Table 4.3: The Goal and General Purposes of the Study ... 79

Table 4.4: The List of the Topics ... 79

Table 4.5: Learning Objectives of the Designed Tests ... 80

Table 4.6: Test Content Description of the Designed Tests... 83

Table 4.7: Task Content Description of the Designed Tests ... 85

xvi  

Table 4.9: Descriptive Statistics of the Participants’ Opinion ... 96

Table 4.10: Revision on the Goal and General Purposes ... 100

Table 4.11: Revision on the Indicators of Learning Objectives ... 101

Table 4.12: Revision on the Texts and Test Content Description... 104

Table 4.13: Revision on the Task Content Description ... 107

Table 4.14: The Description of Participants in the Implementations... 111

Table 4.15: The Description Statistics of the Close-Form Questions On the Implementation ... 112

Table 4.16: The Item Facility (IF) Results of Tipe B ... 117

Table 4.17: The Item Discrimination (ID) Results of Tipe B ... 119

Table 4.18: The B-Index Result of Tipe B ... 120

xvii  

LIST OF FIGURES

Page

Figure 2.1: Kemp’s Instructional Design Model ... 20

Figure 3.1: The adapted R&D cycle combined with adapted Kemp’s Model ... 51

Figure 4.1: Flowchart of the website ... 88

Figure 4.2: Home page display ... 90

Figure 4.3: Tipe A preview using Wondershare QuizCreator 3.2 serial ... 92

Figure 4.4: The mechanism on the use of media ... 94

Figure 4.5: Revision on the website display ... 109  

1  CHAPTER I INTRODUCTION

This chapter is divided into six important parts. They are research background, problem formulation, problem limitation, research objectives, research benefits, and definition of terms.

A. Research Background

As an international language, English is important to communicate and to share information toward people from different countries. Since English becomes a bridge to succeed for any work field, Indonesian government decided to include English language as the compulsory subject in the curriculum within its Competence Standard and Basic Competence (based on the Decree of the Minister of National Education No. 22 Year 2006). In accordance with the School-based Curriculum or Kurikulum Tingkat Satuan Pendidikan (KTSP), teachers select appropriate materials based on Competence Standard (CS) and Basic Competence (BC) guided in Permen No. 22 Tahun 2006.

As cited in Panduan Materi Ujian Nasional Bahasa Inggris SMA/ MA (IPA/IPS/BHS) 2007/2008, National Examination or Ujian Nasional (UN) for English subject is included in written test in the form of multiple-choice test within testing listening and reading skills. The test divided into 15 numbers of listening test and 35 numbers of reading test. Thus, reading test refers to be

 

dominant in the English UN. The focus in reading test consists of text content, rhetoric, structure, and vocabulary.

Moreover, reading is argued as the most essential skill for success in all educational contexts (Brown, 2004: 185). Reading has its function to convey information, to amuse and entertain, to codify, etc. In line with Thompson (1987: 50) reading is an activity integrating the text and readers’ background knowledge to build meaning. It means that readers require other element to find the meaning of the test including their knowledge about grammar, vocabulary, spelling, eye movement, and intellectual comprehension. However, reading skill mastery may facilitate the learners to master the language.

To accomplish the study, the writer focuses on designing a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The reason for conducting the test in senior high school in Kecamatan Cibinong is because students in Kecamatan Cibinong are familiar with computer technology and the Internet, while some schools also provide those facilities to support the students’ need. In fact, the use of computer and the Internet have not been utilized effectively for educational purposes. Based on informal interview with SMA Eka Wijaya teachers, it was found that computer and Internet facilitation are used only for computer course. Besides, English teacher have not explored the use of computer and Internet facility to support English subject.

In this study, the writer applies web-based multiple-choice reading test in order to create a different way in conducting test and to create a learning media

for Grade XII students. Moreover, Grade XII students require various tests to measure their readiness in facing final examination. The web-based test can facilitate students in providing online test which can be assessed anytime without limitation of time and place. The design may create different atmosphere, out of conventional ways, in conducting test. The online process requires connection to internet service provider with a certain address of the website, but students can download the test and print it.

According to Dudeney (2007: 7), the use of technology in the classroom is becoming increasingly important, and it will become a normal part of English language teaching practice in coming year. The writer applies web-based language testing to motivate learners in the use of technology in education. As accessed from Lau Sie Hoe in Web-based Computer-Adaptive Multiple-choice Assessment, computer-based assessment has become increasingly attractive way in educational field. It is proved by the increasing number of students who obtain on-line test via internet because computer-based assessment can reduce cost, and improve the quality and accountability of the assessment process.

This study relates to Information and Communication Technology (ICT) because the design applies application software and the Internet that supports students in conducting a test. In online process, computer should connect to the Internet and find the particular website address. Website and internet are different. As cited in Wikipedia, website is one of the communicated services via internet. Meanwhile, the Internet provides a global data communication system or a hardware and software infrastructure that provide connectivity between

 

computers. As the result, website is only a part of a huge data of the Internet. By using the Internet for conducting the test, students gain benefit in which they can practice everywhere and every time. As accessed on Kurnia Nusa’s in Web-based Distance Learning, students will not be limited by time and space to study; what they need is a computer or a notebook that has connected with the Internet. The test can be accessed easily via Internet that might content test work-sheet display, answer keys display, the score and feedback column. By conducting web-based test, the students do not need the presence of the teacher to provide the test and to assign feedback because one promising aspects of web-based assessment is instant feedback (Peat, 2003:19).

In contrary, computer-based assessment is infrequently conducted in Indonesia especially for senior high school level. Yet, most of Grade XII students of senior high school require some test references to prepare final exam. By conducting a web-based test, the students can do the test outside the class by using Internet facilities from their house or Internet cafés. Besides, the Internet offers chances for students to be familiar with ICT.

Among the various format of test item, multiple-choice item has been used in computer-based assessment due to the ability of computer to be programmed to enable automatic and easy retrieval of test scores and feedback (Conrad et al, 2004). Furthermore, multiple-choice appears as a simple kind of item to construct. Brown (2004: 55) stated that multiple-choice format is practical and reliable for the test makers and test takers. By the automatic and easy retrieval of test score

and feedback, it reduces content misconceptions, reasoning errors, and learning difficulties which often face by students after conducting test (Paul et al, 1994).

The writer focuses on reading test because it is commonly used to evaluate students understanding about language that includes comprehension text, grammar, and vocabulary understanding. Besides, reading takes place among other skills. Wallace (2003: 3) also stated that reading as one of the skills plays an important role in English language. It is because reading is the awareness of the way we used language. Besides, reading is considered difficult by many learners because of all the components and linguistic features, that is why people have difficulties in reading because they have to master all the language components and linguistics features as the same time before they can master the reading skills.

Furthermore, the writer measures student’s reading acquisition through multiple-choice test because it provides benefits for teacher and students from the aspect of practicality. According to Brown (2004: 185), the reading tests do not end with the measurement of comprehension, but also strategy to understand the text. Brown (2004: 194) also stated that multiple-choice test is the most popular method of testing a reading knowledge of vocabulary and grammar, mainly for reasons of practicality. It is easy to administer and can be scored quickly. On the other hand, UN in Indonesia also applies in the form of multiple-choice. In addition, the tests encourage the students and teachers in senior high school as a supplementary test. This web-based test can enrich students and teachers for trying out their knowledge.

 

B. Problem Formulation

Dealing with designing a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong, the writer intended to answer a problem: What is the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong?

C. Problem Limitation

The writer determined some limitations in order to make constraints of this study. First, the study focused on the developing application using software in the form of multiple-choice reading test. The writer decided to design web-based test by using Adobe® Flash® and Wondershare QuizCreatorv 3.2 program. Second, the study employed reading exercises to measure student’s comprehension on the text using web-based test. Third, the study was limited to particular reading materials addressed in School-based Curriculum (KTSP). The tests were based on reading materials which are appropriate with School-based Curriculum, Competence Standard, and Basic Competence.

The test contents followed the CS and BC for Grade XII students of senior high school including narrative text, explanation text, discussion text, and review text. For the test construction, the writer applied multiple-choice test including reading comprehension test and grammatical test. The test covers some micro- and macroskills. For the microskills, the test includes recognizing grammatical world classes, recognizing system, and recognizing particular meaning in different

grammar form. For the macroskills, the test includes detecting main idea, supporting idea, information, generalization, and exemplification; inferring links and connection between the information; distinguishing between literal and implied meanings; and developing reading strategies for interpretation of texts. Each test consists of 35 questions including 5 options which contain of one answer and four distractors.

Here, the subject of the study focused only for Grade XII students of senior high school in Kecamatan Cibinong. The writer decided this level because they had already understood the language components and linguistic features, they are able to read an English text, and thus they can understand direction in order to operate the computer. On the other hand, the design can also be accessed as an exercise for preparing the National Examination because it covers some materials for the national examination such as narrative, explanation, and discussion text. Moreover, the designed test provides some questions that are generally applied in national examination such as reading comprehension questions (main idea, supporting idea, word/ phrase meaning, and true/ false statement) and grammatical questions (connectors and conditional sentence).

D. Research Objectives

The objective of this study was mainly to answer the problem previously stated in the problem formulation that is to develop the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high

 

school in Kecamatan Cibinong. In this study, the writer would like to design and to present the design.

E. Research Benefits

This study was expected to bring about some contributions for students, the teachers and also the other test designers as follows:

1. For students

This study may help students to prepare the national examination especially in reading test. Then, the tests may enhance students’ knowledge of grammar, vocabulary, and reading comprehension. Besides, the students are able to practice the tests without the presence of teacher and students obtain the result immediately. While students conduct the web-based test, they learn a modern type of test and can be familiar with ICT.

2. For teachers

This study may facilitate teachers in conducting different ways of giving a test by using computerized test. Moreover, the teachers can apply the design as a media that carries ICT for teaching English.

3. For other test designers

The study would inform how the designed test was developed based on the theories and the implementation. Then, the designed test can be used to carry out another study on the effectiveness of technology in teaching and learning process especially for Indonesian students.

F. Definition of Terms

In this part, the writer provided the definition of each term so that the readers have the same view and it may reduce misunderstanding.

1. Designing

Designing is defined as drawing plans or planning a system to decide how something will look or work. According to Briggs (et al, 1977), designing is a process of analysis or learning need and goal and the development of a delivery system to meet the needs; includes development of instructional materials and activities; and try out and revision of all instruction and learner assessment activities. The design must be planned and it is developed for specific purposes. In this study, designing is preparing a plan for assessing students’ reading ability in the form of multiple-choice test using computer. The design in this study is a set of web-based multiple-choice reading test. 2. Web-based Test

According to Roever (2001: 84), Web-based Test (WBT) is a Computer-based Language Test which is delivered via the World Wide Web (www). The term Web-based Testing is defined as a small-scale “home-grown” test available on website (Brown, 2004: 14). WBT can be offered any time unlike mass paper and pencil administrations. Moreover, it also provides immediate feedback for the test takers.

Web-based tests share many characteristics of traditional Computer-based Tests (CBT), but using the Web as delivery medium. Moreover, the Web is a system of interlinked hypertext documents accessed via the Internet.

 

In this study web-based test means that the test is provided in the website and conducted using computer as testing medium which is connected via Internet. 3. Multiple-choice

Multiple-choice items are a form of assessment item in which respondents are asked to select one or more of the choices from a list. Multiple-choice item has options where one of those options is a key or correct response and the others are distractors (Brown, 2004:56). In this study, the test-takers select one of five options as a correct answer. The multiple-choice tests focus on formal aspects of language (lexical, grammatical, and few discourse features).

4. Reading Test

According to Brown (2004:185), reading test is a test that measures learners’ ability in mastering fundamental bottom-up strategies (for processing separate letters, words, and phrases) and top-down (conceptually driven strategies for comprehension); and in developing formal schemata (background information and cultural experience). The writer selects the materials based on CS and BC, including narrative text, explanation text, discussion text, and review text in the form of functional text, essay, and monolog. The writer designs selective reading test which focuses on formal aspects of language. This category includes as testing reading comprehension, vocabulary, and grammar.

5. Grade XII Students of Senior High School

Grade XII students of senior high school are the third-year level students in senior high school. Generally, the students have already achieved all materials in senior high level because they are prepared for national examination. Grade XII students often review and tryout materials for the preparation of national examination to drill their knowledge and skill. Consequently, they are able to understand and accustomed to analyze questions in the form of multiple-choice test.

6. Kecamatan Cibinong

Kecamatan Cibinong is a sub-district which becomes the central of governmental activity in Bogor Regency (Kabupaten Bogor), West Java. There are six senior high schools in Kecamatan Cibinong, three state schools and three private schools. They are SMAN 1 Cibinong, SMAN 2 Cibinong, SMAN 3 Cibinong, SMA Mardi Waluya Cibinong, SMA Eka Wijaya Cibinong, and SMA PGRI 1 Cibinong. This study was conducted for students of SMA Eka Wijaya Cibinong, SMA Mardi Waluya Cibinong, and SMAN 1 Cibinong because those schools are favorite schools in Kecamatan Cibinong. Annually, those schools achieved higher range of score of national examination result in Kecamatan Cibinong. Besides, the locations of those schools are nearby the public centre in Kecamatan Cibinong.

     

12 CHAPTER II

REVIEW OF RELATED LITERATURE

This chapter deals with the discussion on the related literatures to support the study. The discussion includes two main subtopics. They are theoretical description and theoretical framework.

A. Theoretical Description

In this part, the writer concerns with the theories which are used to the accomplishment of this study. This part is divided into seven parts. They are the theory of Educational Research and Development (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, and Application Software.

1. Educational Research and Development (R&D)

According to Borg and Gall (1983: 771), Educational Research and Development (R&D) is a process used to develop and to validate educational products and it consists of a cycle in which a version of the product is developed, field-tested, and revised on the basis of field-test data. The goal of Educational Research and Development methodology is to discover new knowledge through the basic research or to answer specific questions about practical problems through applied research (Borg and Gall, 1983:772). In particular, this methodology unites the gap between research and practice by conducting research

and using the result to build test products. In Educational Research and Development, the ten steps used to develop mini-courses are described as follows: a. Research and Information Collecting

This step includes review of literature, classroom observations, and preparation of report of state of the art (Borg and Gall, 1983: 775). Literature review is conducted to collect information that sustain on the design planning. To support the information, classroom observation is conducted to gain information about classroom activity and situation.

b. Planning

It includes defining skills, starting objectives, determining course sequence, and small scale feasibility testing (Borg and Gall, 1983: 775). The focus on this step is determining the skill being improved by considering the objectives of the study that have to be achieved by the product.

c. Developing Preliminary Form of Product

This step includes preparation of instructional materials, handbooks, and evaluation devices (Borg and Gall, 1983: 775). Questionnaire and interview list, as evaluation devices, are designed to obtain information and feedback of the preliminary form of the product. According to Borg and Gall (1983: 781), the procedure of developing the product will differ depending on the nature of the product.

d. Preliminary Field Testing

In this step, the product is applied in 1 to 3 schools, using 6 to 12 subjects. Besides, the designer gathers data through interview, observation, and

 

questionnaire then analyzes the data collected (Borg and Gall, 1983: 775). This step is suggested to test the feasibility of the product before it is implemented. e. Main Product Revision

In this step, the designer revises the product as suggested by the Preliminary Field Test results (Borg and Gall, 1983: 775). The data obtained from interview and questionnaire, are analyzed to improve the product.

f. Main Field Testing

In this step, the revised product is conducted in 5 to 15 schools with 30 to 100 subjects. The data on subjects’ pre-course and post-course performance are collected and analyzed in the form of qualitative data. Then, the results are evaluated with respect to course objectives and are compared with control group data, when appropriate (Borg and Gall, 1983: 775).

g. Operational Product Revision

The designer revises the product as suggested by Main Field Test results (Borg and Gall, 1983: 775).

h. Operational Field Testing

This step has a purpose to determine whether this product is completely ready to be used at schools or not. In this step, the designer conducts the product in 10 to 30 schools involving 40 to 200 subjects (Borg and Gall, 1983: 775). The product has to be tested and revised completely in every part by conducting interview, observation, and questionnaire. The collected data should be analyzed to determine whether the product is complete.

i. Final Product Revision

This step is the final revision of the product as suggested by Operational Field Test result. The final version for the complete package of product is carried out after the data from the operational field test have been analyzed (Borg and Gall, 1983: 786).

j. Dissemination and Implementation

This last step is the technique to adjust the cost of the product developing process by operating effective dissemination and implementation the product to intended users (Borg and Gall, 1983: 786). The designer reports on product at professional meetings and in journals; works with publisher who assumes commercial distribution; and monitors distribution to provide quality control.

In general, the Education Research and Design (R&D) cycle consists of studying research findings pertinent to the product to be developed, developing the product based on those findings, field testing it in the setting where it will be eventually, and revising it to correct the deficiencies found in the field stage.

There is one step called product selection that should be applied before the Educational R&D process. It is necessary to describe as specifically as possible the educational product that is to be developed (Borg and Gall, 1983: 776). The description of product selection should include: (1) an overall narrative description of the proposed product, (2) a tentative outline of what the product will include and how it will be used, and the most important (3) a specific statement of the objectives of the product.

 

The criteria for product selection used include the following:

1) Does the proposed product meet an important educational need?

2) Is the state of the art sufficiently advanced so that there is a reasonable probability that a successful product can be built?

3) Are personnel available who have the skills, knowledge, and experience necessary to build this product?

4) Can the product be developed within a reasonable time?

This planning provides the foundation upon which later revisions are built.

2. Instructional Design Model

In this study, the writer adopts Kemp’s instructional design model to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The writer decided to adopt Kemp’s model because it has a flexible process and can be applied in any educational level. According to Kemp (1977:6) the method can be applied on any educational level: elementary, secondary, or college. In Kemp’s book, he states that the instructional design plan is designed to supply answers to three questions, which may be considered as the essential element of instructional technology (Kemp, 1977:8):

a) What must be learned? (Objectives)

b) What procedures and resources will work best to reach the desired learning level? (Activities and resources)

c) How will we know when the required learning has taken place? (Evaluation)

In order to answer those questions, Kemp forms an instructional material design which consists of eight steps (1977: 13-100) that can be applied to individual topics and then to units and then to complete courses, involving one or few teachers. The eight steps are described as follows:

Step 1: Goal, Topics, and General Purposes

Kemp’s model starts with recognition of the broad goals of teaching learning process. Those goals may be derived from three sources: society, students, and subject areas. The next step is selecting the topics for teaching learning process. Those topics will become the scope of the course program, the basis for the instruction. Then, the next is listing general purposes of teaching learning process.

Step 2: Learner Characteristics

It deals with information about the learner’s capabilities, needs, and interests. According to Kemp (1977: 18-21), there are some factors affecting learner characteristics. They are academic factors, social factors, learning conditions and learning style.

Step 3: Learning Objectives

This part is specifying learning objectives. It is achieved in the terms to measure student behavioral outcomes. The objectives can be grouped into three major categories: cognitive, psychomotor, and affective. In this step, the writer should consider the objectives first before designing the test items. Since the test

 

in this study is a multiple-choice reading test, the writer categorizes the objectives into cognitive group which includes objectives concerning knowledge, comprehension, application, analysis, synthesis, and evaluation (Kemp, 1977: 25). Step 4: List the Subject Content

According to Kemp (1977:43), student’s learning experiences must involve subject content. The content should close-relationship to the objectives and to the student’s needs. In this designed test, the writer listed test contents in line with the learning objectives. However, the test-materials adjust Competence Standard and Basic Competence.

Step 5: Pre-Assessment

The goal of this step is to gain information whether the student prepared to study the topic or unit and the student already competent in some of the stated objectives or not (Kemp, 1977: 50). Pre-assessment is divided into two: prerequisite testing and pretesting. Prerequisite test determines whether students have appropriate background preparation for the topic. While pretesting determines which of the objectives that students may already have achieved. In contrary, this study does not conduct pre-assessment to measure students’ preparation, but it conducted the product verification to measure the feasibility of the design from the experts.

Step 6: Teaching Learning Activities and Resources

Selecting teaching learning activities and resources treat the subject content, so students will accomplish the objectives. After getting the pretest result, a teacher has to determine the most efficient and effective methods and then select

the materials to provide learning experiences that will utilize the content associated with the objectives. The writer skips teaching learning activities because the focus is conducting test. Hence, the writer determined the activity as implementing the web-based test to the students. For the activities of the test, the writer applied three tasks classified into reading comprehension passage, multiple-choice grammar task, and multiple-multiple-choice cloze vocabulary task.

Step 7: Support Services

Teachers need to coordinate the service including funds, facilities, equipments, time, and schedule. In this study, the writer designed a web-based test which required support services such as web-based test program, computer laboratory and the Internet facility.

Step 8: Evaluation

This is the payoff step in instructional design plan for students, teacher, and designer. In this step, the writer evaluates students’ learning in terms of their accomplishment of objectives, with a view to revising and re-evaluating any phase of the plan that need improvement (Kemp, 1977: 91). This step became the important part of the study because the test was implemented to evaluate the designed test and students’ achievement. After implementing the test, the writer analyzed the students’ data through questionnaire and test for the improvement of the design test. The diagram of Kemp’s instructional design model is included in the next page.

 

Figure 2.1: Kemp’s Instructional Design Model (Source: Kemp, 1977:9)

3. Language Testing

In teaching, there are three components that are closely related. They are objectives, teaching activities, and testing. Their relation is shown by the effect of one component to the others. Tests are part of teaching activities where in the teaching process teachers need to observe students’ progresses in learning the materials by giving them some assessment (Brown, 2004: 4). Tests are a subset of an assessment. According to Brown (2004: 3), a test, in the simple form, is a method of measuring a person’s ability, knowledge, or performance in a given

Evaluation Learners’

charac- teristics

Learning objectives

Subject content

Pre-assessment Teaching

learning activities & sources Support

services

Revision

Goal, topics, &

general purposes

domain. First, the method must be explicit and structured to qualify the test. Second, a test should measure general ability, competencies, or objectives. Third, a test measures an individual’s ability, knowledge, or performance. It means that the test could measure student’s personal ability, and then the teacher will know who the students are. Finally, a test measures a domain which means the test shows students’ particular level in the class.

According to Bachman and Palmer (1997: 4), a model of tests on the large scale English Foreign Language tests usually include sections testing grammar, vocabulary, reading, and listening comprehension. These tests measure language ability as a set of components which are grammar, vocabulary, pronunciation, and spelling. Besides to evaluate student’s language ability, language tests can also be used as a tool for clarifying instructional objectives, for evaluating the relevance of the objectives, the materials, and the activities of students following program (Bachman & Palmer, 1997: 4).

a. Types of Language Tests

The basic distinction in language testing involves two families of tests that perform two very different functions (Brown, 2005: 1). First, Norm-referenced Tests (NRT) helps administrators and teachers make program level decision, such as proficiency and placement decision. NRT measures general language ability or proficiencies. It provides score, interpreted in relation to mean, median, standard deviation, and percentile rank (Brown, 2004: 7). Second, Criterion-referenced Test (CRT) helps teacher make classroom-level

 

decision, such as diagnostic and achievement decisions. CRT measures specific objectives based on language points.

Test can be distinguished through some criterions. Based on the procedure, the test can be distinguished as formal test and informal test. According to Brown (2004: 5), informal test can take a number of forms, starting with incidental, unplanned comments and responses, along with coaching and other impromptu feedback to the students. On the contrary, formal tests are exercises or produces specifically designed to tap into a storehouse of skills and knowledge (Brown, 2004: 6). The tests are systematic, planned sampling techniques constructed to give teacher and student an appraisal of student achievement.

Then, test can be characterized as formative and summative test. Formative test evaluates students in the process of forming their competencies and skills with the goal of helping them to continue that growth process (Brown, 2004: 6). The opposite, summative assessment measures and summarizes what a student has grasped, and typically occurs at the end of a course or unit of instruction (Brown, 2004: 6).

In the term of approaches to language testing, test is determined into discrete-point test and integrative test (Oller et al, 1979). Discrete-point test is constructed on the assumption that language can be broke down into its component parts and that those parts can be tested successfully (Brown, 2004:8). In contrast, integrative test measures language competence as a unified set of interacting abilities that cannot be tested separately (Oller et al,

1979). In accordance with the above explanation, this study can be classified as a criterion-referenced test, formal test, summative test, and discrete-point test. b. Test Qualities

Then, to know if the test is effective and useful there are some test qualities according to Brown (2004: 19) such as practicality, reliability, validity, authenticity, and washback. Further, Bachman and Palmer (1997:17) added some qualities such as interactiveness, and impact. Those qualities are explained as follows:

1) Practicality

It means that the test is not expensive, stays within appropriate time constraints, easy to administer, and easy to score. As a consideration, Bachman and Palmer (1997: 36) determined the practicality based on the consideration of the resources that would be required in the test had the balance qualities the teachers want, and the allocation and the management of the resources that were available.

2) Reliability

A reliable test is consistent and dependable. It means that the similar test should bring similar results if it is tested for the same students in a different occasion, yet there are number of factors that may contribute to unreliability of a test those are fluctuations in the student, in scoring, test administration, and the test itself (Mousavi, 2002: 804). Reliability can be considered to be a function of consistency of scores from one set of test and test tasks to another (Bachman and Palmer, 1997: 19). It means the test will not make any

 

difference to a particular test taker whether the test taker takes the test on different occasion and setting.

3) Validity

Validity means that the tests are appropriate, meaningful, and useful in terms of the purpose of the test (Gronlund, 1998: 226). It means that the test really measures what teacher wants to measure, for example: a valid reading test will measure student’s reading ability by asking the students to read and by answering comprehension questions. Moreover, the test designer has to consider the content related evidence, criterion related evidence, construct related evidence, consequential validity, and face validity (Brown, 2004:26-27). The content related evidence means that the content of the test is suitable with the test, for example: if teacher tests speaking, she should give a test in which lets the students to speak. The criterion related evidence is usually related with the results that are supported by the other concurrent performance, and the prediction the test taker’s likelihood of future success. The construct related evidence related with any theory, hypothesis, or model that attempts to indicate the area of language ability teachers want to measure. The consequential validity refers to all the consequences of a test. The face validity refers to the test appearance.

4) Authenticity

Authenticity is the degree correspondence of the characteristics of a given language test task to the features of a target language task (Bachman and Palmer, 1997: 23). The authentic test may present natural language in the test,

contextualized items, relevant and interesting topics for the learners, thematic organization items, and real-world task.

5) Interactiveness

Bachman and Palmer (1997: 25) defined interactiveness as the degree and type of involvement of the test taker’s individual characteristics in accomplishing a test task. The interactiveness means as a quality of any tasks (target language use tasks and test tasks) can potentially vary in test taker interactiveness.

6) Washback or Impact

Washback or impact is the effects of the test for students. It means that a variety of individual will be affected by and thus have an interest, or hold a ‘stake’, in the use of a given test in particular situation Bachman and Palmer (et. al, 2004).

c. Designing Classroom Language Tests

To design classroom language test, a designer requires the purpose of the test, the objectives of the test, the specifications of the test, the selected test task and the separate item arranged, and kind of scoring, grading, and feedback (Brown, 2004: 42). Defining the purpose helps the designer to choose right kind of test. Based on the purpose, test types are distinguished into aptitude tests, proficiency tests, placement tests, diagnostic tests, and achievement tests (Brown, 2004: 43). The explanation is outlined in the next page.

 

1) Aptitude Test

A language aptitude test is designed to measure capacity or general ability to learn a foreign language and ultimate success in that undertaking (Brown, 2004: 43).

2) Proficiency Test

A proficiency test measures overall ability. It has traditionally consisted of standardized multiple-choice items, grammar, vocabulary, reading comprehension, and aural comprehension (Brown, 2004: 44). The tests are almost summative and norm-referenced. The example of this test is Test of English as a Foreign Language (TOEFL®).

3) Placement Test

The test has a purpose to place students into a particular level or section of language curriculum or school (Brown, 2004: 45). The test usually includes a sampling of the material to be covered in the various courses in a curriculum.

4) Diagnostic Test

A diagnostic test is designed to diagnose specified aspects of a language (Brown, 2004: 46). For example, pronunciation test might diagnose the phonological features.

5) Achievement Test

An achievement test is related directly to classroom lessons, units, or even a total curriculum (Brown, 2004: 47). The test has focused on objectives in questions, thus it is limited to particular materials. In this study, the design

is categorized as achievement test, because the test is related directly to curriculum.

4. Reading Skill

This part provides explanation about reading. It includes the nature of reading, reading text types, and reading assessment.

a. Nature of Reading

In teaching language, reading is a skill that teachers expect learners to acquire. It is because reading includes all components of the language such as vocabulary, grammar, and even the pronunciation (Brown, 2004:185). Reading is an activity integrating the text and readers’ background knowledge to build meaning (Thompson, 1987: 50). Moreover, reading is a process of understanding the meaning of written or printed words or symbols. From the definition above, reading is an active process of getting meaning from written text in order to understand and comprehend written form of language.

In learning a language, reading shares with other modes of language, which is listening, speaking and writing (Wallace, 2003: 6). Reading integrated with other skills is used for communication. Furthermore, reading has some purposes in social interaction: for survival, for learning, and for pleasure. In this study, the purpose of reading is for learning. It means that reading serves extending general knowledge of the world. For this purpose, reading is taught in the school because it takes place in academic context. It is also goal oriented.

 

b. Reading Text Types

There are some reading text types taught by students of senior high. They are analytical exposition, anecdote, description, narration, procedure, news item, discussion, explanation, report, recount, review, and hortatory text. Further, Competence Standard and Basic Competence in KTSP mention the goals of study which influence reading text types for English subject. The CS for reading skill for the Grade XII students includes functional texts, essay, and monolog of narrative, explanation, discussion, and review text in daily context for accessing science or knowledge. This study only presents four text types based on competence standard, as follows:

1) Narrative

It has a purpose to entertain, to amuse or to tell the readers about the fiction or non-fiction experience or story that has a problem and the resolution. The generic structure of this type is:

™ orientation: introduction about the characters, places, and times,

™ complication: development of conflicts or problems, and

™ resolution: the reconciliation of the problems.

The dominant language features are using past tense, using action verbs,  chronologically arranged, using conjunctions (then, before, after, soon, etc), and using direct and indirect sentences.

2)Explanation

Explanation text has purpose to explain the processes involved in an activity of natural phenomenon, science, socio-cultural, etc.

The generic structure of this type includes:

™ general statement: general explanation,

™ explanation: a sequenced explanation of why or how something occurs, and

™ closing.

Moreover, explanation text has some dominant language features such as using Simple Present Tense, using action verbs, using passive voice, using complex sentences, using noun phrases (e.g. the large cloud), using adverbial phrases, using technical terms, using general and abstract noun (e.g. the temperature), and using conjunction of time and cause-effect.

3)Discussion

Discussion text is aimed to present an issue overseen by two point of views (pros and cons) before it ends on certain conclusions or recommendations. The generic structure of discussion text is presented as follows:

™ issue: involves the statement of issue and preview,

™ arguments of pros: include several main ideas and elaborations of supporting opinions from the issue,

™ arguments of cons: consist of statements of various viewpoints that oppose the issue and its elaborations, and

™ conclusion or recommendation.

The dominant language features in discussion text is using Simple Present Tense. Besides, the text includes general nouns to maintain the category of issue such as cigarette, uniform, alcohol, etc. The text also uses relating verbs

 

to present information about discussed issue such as smoking is harmful, homework is necessary, etc. Discussion text conveys thinking verbs to express personal views (e.g. believe, feel, hope); involves modalities (e.g. perhaps, must, should, could be); uses adverbials of manner (e.g. deliberately, hopefully); includes detail noun groups to obtain information (e.g. the dumping of unwanted kittens); and uses additives, contrastive, and causal connectives to join the arguments (e.g. however, on the hand, in contrary).

4)Review

This type is purposed to critique or evaluate an art work or event (e.g. movies, show performance, books, etc) for public. The generic structure of this type is:

™ orientation: introduction of the title, characters, the author, etc.,

™ evaluation: critique on several point,

™ interpretation,

™ evaluation, and

™ evaluative summation.

The text generally focuses on specific participants. The text also uses adjectives to point out attitudes (e.g. bad, good for teenagers, interesting), long and complex clauses, and metaphor.

c. Reading Assessment

Reading, as necessary skill, remains a skill of paramount importance as we create assessments of general language ability (Brown, 2004: 185). Therefore, in assessing English language, people use reading for granted. Assessing reading does not end with the measurement of comprehension, but

also strategic to full understanding including fundamental bottom-up and top-down strategies. Bottom-up strategies begins with processing separate letters, words, phrases, clauses and then sentences of the text. Bottom-up strategies helps the readers to catch and understand the meaning of information of the text. Whereas top-down strategies, driven strategies for comprehension, consists of background information, cultural experience, life experience, and background knowledge that helps the reader to carry out interpretation of the text. Moreover, the test maker should also consider a number of different types or genres of written texts, the components of reading ability, and specific task. 1) Genres and Types of Reading Assessment

According to Brown (2004: 186), there are three specifications of common genres for assessments of reading ability. They are academic reading, job-related reading, and personal reading. In this study, the writer considers academic reading. The genre of text allows readers to apply certain representation that assist in extracting appropriate meaning.

For considering test procedures, several types of reading performance are typically identified. They are reading perceptive, reading selective, reading interactive, and reading extensive (Brown, 2004: 189). This study can be categorized as selective reading test and interactive reading test because it contains of multiple-choice cloze vocabulary/ grammar task and reading comprehension questions.

 

2) Microskills, Macroskills, and Strategies for Reading

The skills and strategies for accomplishing reading appear as crucial consideration in the assessment of reading ability. Brown made the specific skill more detail into micro- and macroskills based on fundamental top-down and bottom-up strategies. Brown (2004: 187) represented micro- and macroskills for considering the objectives.

Microskills for reading comprehension are represented as follows:

a) Discriminate among the distinctive graphemes and orthographic patters of English.

b) Retain chunks of language of different lengths in short-term memory. c) Process writing at an efficient rate of speed to suit the purpose.

d) Recognize a core of words, and interpret word order patterns and their significance.

e) Recognize grammatical word classes (verb, noun, etc), system (tense, agreement, pluralization, etc), patterns, rules, and elliptical forms.

f) Recognize that a particular meaning may be expressed in different grammatical forms.

g) Recognize comprehensive devices in written discourse and their role in signaling the relationship between and among clauses.

Macroskill for reading comprehension are represented as follows:

a) Recognize the theoretical forms of written discourse and their significance for interpretation.

b) Recognize the communicative functions of written texts, according to form and purpose.

c) Infer context that is not explicit by using background knowledge.

d) Infer links and connection between the information and detect such relations as main idea, supporting idea, new information, given information, generalization, and exemplification.

e) Distinguish between literal and implied meanings.

f) Detect culturally specific references and interpret them in a context of the appropriate cultural schemata.

g) Develop and use battery of reading strategies for interpretation of texts. Furthermore, according to Brown (2004: 188), there are some principal strategies for reading comprehension as follows:

a) Identify the purpose in reading a text.

b) Apply spelling rules and conventions for bottom-up decoding. c) Use lexical analysis to determine meaning.

d) Guess at meaning when you aren’t certain. e) Skim the text for the gist and for main ideas. f) Scan the text for specific information.

g) Use silent reading techniques for rapid processing.

h) Use marginal notes, outlines, charts, or semantic maps for understanding and retaining information.

i) Distinguishing between literal and implied meanings. j) Capitalize on discourse markers to process relationships.

 

In this study, the test covers some micro- and macroskills. For the microskills, the test includes recognizing grammatical world classes, recognizing system, and recognizing particular meaning in different grammar form. Those microskills are implemented in grammatical questions. For the macroskills, the test includes considering reading comprehension strategies; detecting main idea, supporting idea, information, generalization, and exemplification; inferring links and connection between the information; distinguishing between literal and implied meanings; and developing reading strategies for interpretation of texts. Those macroskills are implemented in reading comprehension questions.

5. Designing Multiple-choice Items

Multiple-choice appears as a simple kind of item to construct, but it is difficult to design correctly. Multiple-choice test are also commonly used in the elementary, junior and high school in order to test students’ achievement toward their study especially for summative test. However, multiple-choice formats are practical and reliable (Brown, 2004: 55). The practicality to administer and to score become the reason of why multiple-choice become the favorite test used in final examination.

In contrary, according to Hughes (2003: 76-78), there are a number of weaknesses of multiple-choice items. The first, technique test only recognizes knowledge. The multiple-choice test has limitation in measuring some skills and ability. For example: it cannot assess speaking ability. The second, guessing may

have a considerable effect on the test scores. If the student does not know the answer, they consider guessing the options. The answer can be right depending on their luck. Then, it is very difficult to write successful items because the teacher should consider distractor efficiency. The washback may be harmful for the students because the test is restricted what can be tested. Then, cheating may be facilitated since the students only mention the letter A, B, C, D, or E.

According to Brown (2004: 56), a multiple-choice format has a primer terminology. The first, multiple-choice items are all receptive or selective. It means that the test-takers can choose from a set of responses or usually called supply. The second, multiple-choice has a stem that presents a stimulus and options or alternatives to choose from. The third, one of those options is key (correct response) and the others are distractors.

To make appropriate multiple-choice items, teachers should consider guidelines for designing multiple-choice items. In the following, there are four guidelines for designing multiple-choice items (Gronlund, 1998: 60).

a. Design each items to measure a specific objectives.

In this step, the designer should consider or revise the items so that can be appropriate with the comprehension question.

b. State both stem and options as a simply and directly as possible.

It can help students to identify the test issue without distracting the students unnecessarily. The good item is to get directly to the point.

 

c. Make certain that the intended answer is clearly the only correct one.

The designer has to eliminate the unintended possible answer so the test takers will not be confused in answering the question.

d. Use items indices to accept, discard, or revise items.

To measure whether the items are appropriate and suitable, teachers can conduct three indicators: item facility, item discrimination, and distractor analysis (Brown, 2004: 58).

In contrary, CRT development and improvement projects is quite different because the purpose of CRT is to assess how much objective has been learned by each student, CRT assessment has to occur before and after instruction in order to determine whether there was any gain in sources (Brown, 2005: 79). CRT involves administering Item Facility (IF), Difference Index (DI), and B-Index (Brown, 2005: 80). However, the process of gaining information about DI is consuming time, so the writer replaced DI with Item Discrimination (ID) to analyze the MC items.

The IF is extent to which an item is easy or difficult for the proposed group of the test takers and the appropriate test items generally have IFs range between 0.15 and 0.85 (Brown, 2004: 58). The ID indicates the power of discriminating for each item. In line with Brown (2004:59), high discriminating power would approach a perfect 1.0, and no discriminating power would be 0.0. B-Index is an item statistic that compares IFs of those students who passed a test with the Ifs of those who fail it (Brown, 2005:82). It indicates the degree to which the masters outperformed the non-masters on each item.

6. Web-based Test

In recent years, the advancement of technology, particularly the Internet, has affected in the way of people learn. Through the Internet, students are urged to research, explore and express themselves in ways which are not possible, practical or powerful with more conventional means (Castro et al, 1999). The term web-based testing is defined as a small-scale “home-grown” test available on website (Brown, 2004: 14). Besides, the large-scale test which is standardized, involves thousand or even tens thousands of test-takers such as TOEFL® and TOEIC®. Web-based test is also known as computer-assisted or computer-based test.

a. The Advantages of Web-based Test

The use of computer in giving web-based test offers several advantages. Below, they are the advantages according to Brown (2004: 14):

1) self-directed testing on various aspects of a language (vocabulary, grammar, discourse, one or all of the four skills, etc),

2) practice for upcoming high-stakes standardized tests,

3) large-scale standardized test that can be administered easily to thousand of test takers at many different stations, then scored electronically for rapid reporting of result, and

4) it provides opportunity to students for increasing their technological knowledge (Chafe et.al, 1999).

Moreover, the use of internet for web-based test provides more opportunity for the test- takers without the limitation of time and place. For

 

students, WBT can increase motivation levels for educational tasks because it involves learning computer. Muehleisen (et al, 1997) also states that computer utilizing provides a strong intrinsic motivation for students as they are still relatively new and exciting. CBT has flexibility to be offered to the students. It will suit the students who are ready to take the test. Besides the advantages, CBT also has disadvantages such as lack of security and the possibility of chatting, less of human interaction especially in oral production.

b. Role of Computer and Teacher in Web-based Test

The use of computers in testing is attracted because it allows some application software which provides different experiences of conducting a test than conventional paper-pencil test. Computer in WBT is developed to drill students by providing some tests, useful feedback, and direct result. The test items in WBT are made on the basis of students’ level. According to Warschauer (1998), the role of computer is divided into 3. The first, computer can be act as a tutor. It means that the computer replaces the teacher position which provides information for students. Second, it performs as a stimulus which motivates students to learn more. Third, it performs as a horse work. It means that teacher and learners use computer to create information. Since CALL can be used as facilitator for learning languages, computer can also be used to give a test, Computer-based Testing (CBT). In CBT, computers are operated as a testing medium because it facilitates the students to conduct the test. In this study, computer can be categorized as a stimulus because it helps students to learn from their mistakes by using feedback provided.

Teacher’s position in conducting WBT is not directly needed because teacher provides the tests which can be implemented anytime and anywhere through the Internet provider. Although WBT does not need the presence of teacher, he or she has a responsibility to be a facilitator in school by preparing the school facilitation. Means and Olson (et al, 1997) state that there is a certain amount of overhead that goes with learning to use any new technology. It seems that teachers need to acquire skills of operating product of technology in order to provide better education.

c. Website Evaluation

To develop a good website, designer requires considering some criteria. Website development should be user-centered, hence evaluating the evolving design against user requirements based on the objectives of the website design. As assessed in (http: www.usabilitynet.org), the evaluation of the website can be measured from site structure and content, support navigation, page design, management and maintenance, as follows:

1) Structure and content

The substance and information on the site should be relevant to the site and should target the area of the public that the website is concerned with. 2) Support navigation or usability

The site should be user-friendly, with the interface and navigation simple and reliable.

Lalu, lihat konteks kalimatnya: ‘ketika sapi telah…….terlalu tua, ibunya menyuruh  Jack ke pasar dan menjualnya. ‘ 

Yang sesuai adalah option B, artinya: tumbuh.   

26. The appropriate word for number (26) is………… 

a. woke 

b. grown 

c. opened  

d. felt 

e. persuaded 

ANSWER: E 

Perhatikan kalimat soal: 

‘The butcher told the boy that the beans were of great value and (26)………… the silly  fad to sell the cow for the beans.’ 

Kalimat tersebut merupakan kalimat compound sentence/ kalimat setara karena  terdapat conjuction ‘and’. 

Yang hilang dari kalimat tersebut adalah Verb past tense. 

Kata yang sesuai dengan konteks: ‘penjagal memberitahukan si anak laki‐laki bahwa  biji‐bijian tersebut sangat bernilai dan (dia)………iseng saja menjual sapi untuk biji‐ bijian.  

Pilihan yang sesuai adalah E, persuaded: membujuk.   

27. The appropriate word for number (27) is………… 

a. woke 

b. grown 

c. opened  

d. felt 

e. persuaded 

ANSWER:  

Perhatikan kalimat soal:  

He……… the window and jumped to the beanstalk which ran up just like a big ladder.  (merupakan kalimat past tense setara dengan kata penghubung ‘and’) 

Dari konteksnya, pilihan yang sesuai adalh C, opened: membuka. 

Dia membuka jendela dan lompat ke tangkai tanaman kacang yang menjulang  seperti tangga besar. 

 

For grammar questions: 

Read the questions carefully and select the appropriate answer from the five  options (A,B,C,D, and E) by clicking the bullet in front of the letter! 

 

28. Jimmy was shocked as thought…  a. he was seeing a ghost  b. he had seen a ghost  c. he has seen a ghost  d. he was a ghost  e. he sees a ghost 

 

Merupakan kalimat conditional sentence tipe 3.  

Jimmy was shocked as thought…. (kalimat selanjutnya harus diikuti past perfect)  Past perfect: S + had + V3 

 

29. Could you please tell me… 

a. where is the Sanata Dharma administrative office?  b. where the Sanata Dharma administrative office?  c. where the Sanata Dharma administrative office is?  d. where did the Sanata Dharma administrative office?  e. where was the Sanata Dharma administrative office?  Answer: C 

Yang merupakan kalimat Tanya adalah “Could   you   please tell me….” sedangkan  kalimat berikutnya bukan kalimat tanya hanya saja mengandung unsur yang ingin  ditanyakan.  

Jawaban yang paling tepat adalah C.   

30. A: What happened to you skirt?  B: Oh, ….when I jumped over the fence.  a. It was torn 

b. To be torn  c. It tore 

d. Tearing 

e. Be torn  ANSWER: A 

Konteks percakapan dalam bentuk past, dan yang dibutuhkan adalah kalimat utama  yang membutuhkan S + Verb. 

Kalimat tersebut merupakan kalimat passive stative dimana objectnya tidak jelas,  sedangkan diikuti adjective yang menjelaskan subjectnya. 

 

31. A: Have the boys had their breakfast?   B: Not yet, they…. 

a. are still talking a bath  b. have taken a bath  c. will take a bath  d. were taking a bath  e. would still take a bath  ANSWER: A 

Pertanyaan merupakan present perfect tense, menanyakan apakah mereka sudah  sarapan; dan jawabannya belum. Yang ditanyakan merupakan alasan mengapa  mereka belum sarapan. Sehingga konteks jawaban yang paling tepat adalah A:  present continues. 

 

32. If the plane had taken off on time, we wouldn’t have been late for the opening of  the seminar. 

This means that we were late because the flight…  a. were delayed 

c. was going to be delayed  d. had not been delayed  e. was not delayed  ANSWER: A 

Pernyataan  di  atas merupakan  bentuk  conditional  sentence  type  3, sehingga  faktanya dalam bentuk past tense dan berkebalikan arti dengan bentuk condisional  sentence.   

 

33. The dollar exchange rate isn’t too….…just now. Let’s change our money next week.   a. faithful 

b. favorable 

c. favor  d. favorite  e. fantasy   Answer: B 

“Nilai tukar dolar sedang tidak..(bagus)..sekarang. Mari kita menukar uang kita  minggu depan.” Jawaban yang paling sesuai B, favorable: bagus/ baik. 

faithful  : setia 

favor  : kebaikan 

favorite  : favorit  fantasy   : fantasi   

34. After seeing the movie Twilight, …….  

a. the reading of the book interested many people.  b. many people want to read the book. 

c. the book made many people want to read.  d. the book was read by many people.  e. many people read many book.  Answer: B 

“Setelah menonton film Twilight,...”  

Jawaban yang paling sesuai adalah B, (banyak orang yang ingin membaca buku  tersebut). 

 

35. Corry  : When did someone build the Suez Canal?  Jerry  : I don’t know for sure, but you can google it.  The PASSIVE for Corry’s question is………. 

a. When was the Suez Canal built? 

b. When was someone building the Suez Canl?  c. When did the Suez Canal built? 

d. Was someone built the Suez Canal?  e. Was the Suez Canal build by someone?  ANSWER: A 

Bentuk kalimat tanya passive (bentuk Past tense) biasanya menggunakan was/ were  setelah kata tanya. 

     

                       

 

ww

ww.tryyoutbahasa

aingg

 

ix

 

ABSTRACT

Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice

Reading Test for the Grade XII Students of Senior High School in Kecamatan

Cibinong. Yogyakarta: English Language Education Study Program, Sanata

Dharma University.

Reading is one of the essential skills for success in all educational contexts because it has function to convey information, to entertain, and to codify. Reading skill may facilitate the learners to master the language. It is because reading integrates the text and the readers’ background knowledge to build meaning. Hence, reading is often implemented and refers to be dominant in a language test. In this study, the writer considered using web-based test in order to create a different way in conducting test. Besides, the web-based test can be accessed by many students without limitation of time and place.

This study is aimed to design a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong. The writer intended to answer a problem: What is the appropriate design of a set of web-based multiple-choice reading test for Grade XII students of senior high school in Kecamatan Cibinong?

Dealing with the design, the writer applied theories related to theory of Educational Research and Development (R&D), Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, and Application Software. Moreover, the design was performed step by step according to the R&D cycle combined with Kemp’s model. They are (1) Learners’ Characteristics and Needs, (2) Goal, General Purposes, and Topics, (3) Learning Objectives, (4) Test Content, (5) Activities and Sources, (6) Support Services, (7) Feedback of the Preliminary Designed Test, (8) Revision of Preliminary Designed Test, (9) Feedback from the Implementation, and (10) Final Test Revision.

The appropriate design in the form of web-based multiple-choice reading test is based on data obtained from a research. The writer developed the website address at www.tryoutbahasainggris.co.nr. The writer utilized application software such as Adobe® Flash® and Wondershare QuizCreatorv 3.2 serial. The website includes seven main pages. They are index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html and link.html. The designed test contains of four topics: narrative, explanation, discussion, and review texts. The test content is divided into two groups: reading test and grammatical test. For the activities, the writer implemented three tasks classified into reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Each test consists of 35 questions, 5 options and 100 minutes time constraint. The result of the implementation of designed test showed that the average mean of the whole data was 3.77 fromthe scale 5.0.

In conclusion, the implementation result strengthens that the test was appropriate and acceptable for the Grade XII students of senior high school in

x ABSTRAK

Lestari, Antonia Fitri. 2011. Designing a Set of Web-based Multiple-choice

Reading Test for the Grade XII Students of Senior High School in Kecamatan

Cibinong. Yogyakarta: Program Studi Pendidikan Bahasa Inggris, Universitas

Sanata Dharma.

Membaca merupakan salah satu keahlian yang bagi keberhasilan dalam semua konteks pendidikan karena memiliki fungsi untuk menyampaikan informasi, untuk menghibur, dan untuk menyusun kata-kata. Keterampilan membaca dapat memfasilitasi peserta didik untuk menguasai bahasa. Hal ini dikarenakan membaca menggabungkan teks dengan latar belakang pengetahuan pembaca untuk membangun makna. Oleh karena itu, membaca seringkali dilaksanakan dan menjadi acuan utama dalam sebuah test bahasa. Dalam penelitian ini, penulis mempertimbangkan untuk menggunakan tes berbasis web untuk menciptakan cara yang berbeda dalam melakukan tes. Selain itu, tes berbasis web dapat diakses oleh siswa tanpa batasan waktu dan tempat.

Penelitian ini bertujuan untuk merancang satu set multiple-choice reading

tes berbasis web untuk siswa kelas XII SMA di Kecamatan Cibinong. Penulis bermaksud untuk menjawab suatu permasalahan, yaitu: seperti apa rancangan satu set multiple-choice reading tes berbasis web yang sesuai untuk siswa kelas XII SMA di Kecamatan Cibinong.

Berhubungan dengan rancangan tersebut, penulis menggunakan teori-teori yang berkaitan dengan metode Penelitian dan Pengembangan (R&D),

Instructional Design Model, Language Testing, Reading Skill, Designing Multiple-choice Items, Web-based Test, dan Aplikasi Software. Selain itu, desain menggunakan langkah-langkah sesuai dengan siklus R&D dikombinasikan dengan model design Instruksi Kemp. Langkah-langkah yang digunakan yaitu (1) Karakteristik dan kebutuhan siswa, (2) Tujuan umum dan topik, (3) Tujuan pembelajaran, (4) Konten uji, (5) Kegiatan dan sumber, (6) Layanan dukungan, (7) Komentar dari rancangan awal tes, (8) Revisi rancangan awal tes, (9) Komentar dari implementasi tes, dan (10) Revisi dari final tes.

Pembuatan tes didasarkan pada data yang diperoleh dari penelitian. Penulis mengembangkan website dengan alamat www.tryoutbahasainggris.co.nr. menggunakan aplikasi Adobe® Flash ® dan Wondershare QuizCreatorv 3.2. Situs ini mencakup 7 laman, yaitu: index.html, panduan.html, tentang.html, tipea.html, tipeb.html, tipec.html dan link.html. Rancangan tes terdiri dari 4 topik yaitu

narrative, explanation, discussion, dan review. Isi tes ini dibagi menjadi dua yaitu

reading tes dan gramatikal tes. Untuk kegiatan dalam tes, penulis menerapkan tiga jenis tugas yaitu reading comprehension passage, multiple-choice grammar task, and multiple-choice cloze vocabulary task. Setiap tes terdiri dari 35 pertanyaan, 5 pilihan jawaban dan waktu pengerjaan 100 menit. Hasil pelaksanaan tes menunjukkan bahwa nilai rata-rata dari seluruh data adalah 3,77 dari skala 5,0.

Sebagai kesimpulan, hasil implementasi memperkuat bahwa tes tersebut sudah sesuai dan dapat diterima bagi siswa kelas XII SMA di Kecamatan Cibinong.