The design of an achievement multiple-choice item narrative reading test for the tenth grade Pangudi Luhur Santo Yosef Surakarta Senior High School.

(1)

THE DE NARRAT LUHU EN DEP FAC

ESIGN OF A TIVE READ UR SANTO Y

Presented to O GLISH LA PARTMEN CULTY OF SA AN ACHIE DING TEST YOSEF SU A

as Partial F Obtain the Sa in English

Ru Student N

ANGUAGE E NT OF LAN

F TEACHER ANATA DH

EVEMENT M T FOR THE URAKARTA A THESIS Fulfillment arjana Pend Language E By uth Widyasar Number: 061 EDUCATIO NGUAGE AN RS TRAINI HARMA UN GYAKART 2011 MULTIPLE E TENTH G A SENIOR H

of the Requ didikan Deg Education

ri 1214020

ON STUDY ND ARTS E ING AND E NIVERSITY TA E-CHOICE GRADE PA HIGH SCH uirements ree Y PROGRA EDUCATIO EDUCATIO Y E ITEM ANGUDI HOOL M ON ON

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(2)

THE DE NARRAT LUHU EN DEP FAC

ESIGN OF A TIVE READ UR SANTO Y

Presented to O GLISH LA PARTMEN CULTY OF SA AN ACHIE DING TEST YOSEF SU A

as Partial F Obtain the Sa in English

Ru Student N

ANGUAGE E NT OF LAN

F TEACHER ANATA DH

of the Requ didikan Deg Education

ri 1214020

ON STUDY ND ARTS E ING AND E NIVERSITY TA E-CHOICE GRADE PA HIGH SCH uirements ree Y PROGRA EDUCATIO EDUCATIO Y E ITEM ANGUDI HOOL M ON ON

(3)

S C THE DE NARRAT LUHU Sponsor Caecilia Tut

ESIGN OF A TIVE READ UR SANTO Y

tyandari, S.P A AN ACHIE DING TEST YOSEF SU Ru Student N A Pd., M.Pd. ii A Thesis on

EVEMENT M T FOR THE URAKARTA By uth Widyasar Number: 061 Approved by MULTIPLE E TENTH G A SENIOR H

ri 1214020 y E-CHOICE GRADE PA HIGH SCH

23 May 20

E ITEM ANGUDI HOOL

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(4)

(5)

Dedication Page

This thesis is proudly dedicated to:

My beloved parents Bernadheta Sri Rejeki and Kristian Budi Santoso, S.E My dearest husband Gede Manuel Kris Sudianto

My beloved brother Thomas Tirta Suminar

All of my friends: Ria, Manda,Dewati, Tiwi, and all of the 2006 students of English Education & Study Program

The readers of this thesis

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(6)

STATEMENT OF WORK’S ORIGINALITY

I honestly declare that this thesis, which I have written, does not contain the work or parts of the work of other people, except those cited in the quotations and references, as a scientific paper should.

Yogyakarta, 7 June 2011 The Writer

Ruth Widyasari 061214020

(7)

LEMBAR PERNYATAAN PERSETUJUAN

PUBLIKASI KARYA ILMIAH UNTUK KEPENTINGAN AKADEMIS

Yang bertanda tangan di bawah ini, saya mahasiswa Universitas Sanata Dharma: Nama : Ruth Widyasari

Nomor Mahasiswa : 061214020

Demi pengembangan ilmu pengetahuan, saya memberikan kepada Perpustakaan Universitas Sanata Dharma karya ilmiah saya yang berjudul:

THE DESIGN OF AN ACHIEVEMENT MULTIPLE-CHOICE ITEM NARRATIVE READING TEST FOR THE TENTH GRADE PANGUDI LUHUR SANTO YOSEF SURAKARTA SENIOR HIGH SCHOOL

Dengan demikian saya memberikan kepada Perpustakaan Universitas Sanata Dharma hak untuk menyimpan, mengalihkan dalam bentuk media lain, mengelolanya dalam bentuk pangkalan data, mendistribusikan secara terbatas, dan mempublikasikannya di Internet atau media lain untuk kepentingan akademis tanpa perlu meminta ijin dari saya maupun memberikan royalty kepada saya selama tetap mencantumkan nama saya sebagai penulis.

Demikian pernyataan ini yang saya buat dengan sebenarnya. Dibuat di Yogyakarta

Pada tanggal: 7 Juni 2011 Yang menyatakan

Ruth Widyasari

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(8)

vii ABSTRACT

Widyasari, Ruth. 2011. The Design of an Achievement Multiple-Choice Item Narrative Reading Test for the Tenth Grade Pangudi Luhur Santo Yosef Surakarta Senior High School. Yogyakarta: English Language Education Study Program, Sanata Dharma University.

This research focused on designing a multiple-choice item reading test for the tenth grade Pangudi Luhur St. Yosef Surakarta Senior High School 2009/2010 academic year. Designing a good multiple-choice reading test was very challenging since the multiple-choice tests are extremely difficult to design correctly. In this research, the writer designed a multiple-choice reading test as an achievement test. The writer limited the scope of reading as a skill to be tested into a genre so–called narrative. Narrative has been taught in the tenth grade of senior high school. Furthermore, the writer also investigated the validity of the test by analyzing the test items and giving questionnaire to the English teacher who taught the tenth grade Pangudi Luhur St. Yosef Surakarta Senior High School. The aims of this research were: (1) to describe how the multiple-choice item reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School is designed; (2) to present the multiple-choice item reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School.

In addition this research was done by applying five steps of Borg’s and Gall’s R & D (Research & Development) cycle: research and information collecting, planning, developing preliminary form of product, preliminary field testing, main product revision and combined those steps with Bachman’s and Palmer’s test development which consisted of three stages: design, operationalization and administration. By combining the five steps of R & D cycle and the three stages of test development, the writer were able to describe how to design the multiple-choice item reading test and present the multiple-choice item reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School.

The test consisted of five (5) narrative reading texts taken from the internet and twenty (20) questions with five (5) items. The writer distributed the test to the tenth grade students and collected the answer sheets. From the answer sheets then the writer investigated the construct validity of the test by counting the Item Facility (IF) and Item Discrimination (ID). The writer also collected the result of the questionnaire from the English teacher who taught the tenth grade students to investigate content validity and face validity. After analyzing the validity of the test, the writer revised the test. There were 3 (three) numbers that the items should be revised because of very low point of item facility and item discrimination which meant the construct validity was not good.

In conclusion, the writer applied the five steps of Borg’s and Gall’s R & D cycle and three stages of Bachman’s and Palmer’s test development which helped the writer to design the test correctly. By doing five steps of R & D cycle and

(9)

viii

three stages of Bachman’s and Palmer’s test development, the problem formulations could be answered. Furthermore, after the revision was made, the test could be applied in the tenth grade students of senior high school.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(10)

ix ABSTRAK

Widyasari, Ruth. 2011. The Design of an Achievement Multiple-Choice Item Narrative Reading Test for the Tenth Grade Pangudi Luhur Santo Yosef Surakarta Senior High School. Yogyakarta: Program Studi Pendidikan Bahasa Inggris, Universitas Sanata Dharma.

Penelitian ini difokuskan pada perancangan tes membaca dalam bentuk pilihan ganda untuk kelas X (sepuluh) Sekolah Menengah Atas Pangudi Luhur St. Yosef Surakarta tahun ajaran 2009/2010. Penulis memilih tes pilihan ganda dalam penelitian ini karena jarang digunakan untuk tes kemampuan akademik. Sebagai tes kemampuan akademik, penulis perlu mempertimbangkan kurikulum dan silabus sebagai pedoman untuk pembuatan tes. Penulis membatasi ruang lingkup membaca sebagai keterampilan yang akan diuji pada bentuk teks narasi. Teks narasi telah diajarkan di kelas X SMA. Selanjutnya, penulis juga meneliti validitas tes tersebut dengan menganalisa pilihan ganda dari tes dan memberikan kuesioner kepada guru Bahasa Inggris yang mengajar kelas X SMA Pangudi Luhur St Yosef Surakarta. Tujuan penelitian ini adalah: (1) untuk menjelaskan bagaimana tes membaca dalam bentuk pilihan ganda untuk siswa kelas X Pangudi Luhur St Yosef didesain; (2) untuk menyajikan tes membaca dalam bentuk pilihan ganda untuk siswa kelas X SMA Pangudi Luhur St Yosef Surakarta.

Lebih Jauh lagi, penelitian ini dilakukan dengan menerapkan lima langkah siklus R & D Borg dan Gall: pengumpulan informasi dan materi penelitan, perencanaan, pengembangan bentuk awal dari produk, pengujian tes awal, revisi produk utama dan dikombinasikan dengan pengembangan tes oleh Bachman dan Palmer yang terdiri dari tiga tahap: desain, operasionalisasi dan administrasi. Dengan menggabungkan lima langkah siklus R & D dan tiga tahap pengembangan tes, penulis mampu menjelaskan bagaimana merancang tes membaca dalam bentuk pilihan ganda dan menyajikan tes membaca dalam bentuk pilihan ganda untuk siswa kelas X SMA Pangudi Luhur St Yosef Surakarta.

Tes ini terdiri dari lima (5) teks bacaan narasi yang diambil dari internet dan dua puluh (20) pertanyaan dengan lima (5) pilihan. Lebih jauh lagi, penulis

(11)

memberikan tes tersebut kepada siswa kelas X dan mengumpulkan lembar jawaban dari tes tersebut. Dari lembar jawaban tersebut penulis mencari validitas konstruk melalui penghitungan item fasilitas dan item diskriminasi. Penulis juga mengumpulkan hasil kuesioner dari guru bahasa Inggris yang mengajar siswa kelas X untuk menyelidiki validitas isi dan validitas bentuk. Setelah menganalisa validitas tes, penulis merevisi test tersebut. Ada tiga nomor yang harus direvisi karena rendahnya poin item fasilitas dan item diskriminasi yang berarti validitas konstuknya tidak bagus.

Sebagai kesimpulan, penulis mengaplikasikan lima langkah dalam siklus R & D oleh Borg dan Gall serta tiga tahap dati pengembangan tes oleh Bachman dan Palmer yang membantu penulis mendesain tes dengan benar. Dengan melakulan lima langkah R & D sera tiga tahapan dari pengembangan tes, rumusan masalah telah terjawab. Lebih jauh lagi, setelah direvisi, tes tersebut dapat diaplikasikan untuk kelas sepuluh SMA.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(12)

ACKNOWLEDGEMENTS

In completing this thesis, I did not work alone. Some people helped me in finishing this thesis. Those people gave me supports and guidance when I wrote the thesis and did the research. In this occasion, I would like to express my gratitude to some special people.

First of all, my sponsor, Caecilia Tutyandari, S.Pd., M.Pd. who always patiently gave the writer guidance in writing this thesis and doing the research and support in doing this thesis from the beginning until the writer could finish it. I still remember how I started writing this thesis enthusiastically and in the ninth semester, I did nothing. Finally, I did it. I am grateful for the assistance during finishing my thesis.

My special gratitude goes to the principal of St. Yosef Pangudi Luhur Surakarta Senior High School, Br. Agustinus Mujiya, S.Pd. and all of the teachers in Pangudi Luhur St. Yosef Surakarta Senior High School who gave me opportunity to do my research. My special thankfulness goes to Maria Olise C. H., S. Pd. as the English teacher of the tenth grade Pangudi Luhur St. Yosef Senior High School 2009/2010 academic year who always supported me in doing the research and finishing this thesis and also became a partner to discuss everything.

My thanks are also for the tenth grade students of Pangudi Luhur St. Yosef Surakarta Senior High School 2009/2010 academic year. All of the students were doing a great job to help the writer finishing the research.

My deepest gratitude goes to my beloved parents, Bernadheta Sri Rejeki and Kristian Budi Santoso, S.E., my brother, Thomas, and dearest husband,

(13)

xii

Manuel. Those people I love so much never stopped asking me to finish this thesis as soon as possible. I really appreciate their love, support, and prayers.

My best gratitude is for all of my friends and all of my lecturers. The contribution of those people to my study is very precious for me. I learn many things from them.

Finally,” May He give you the desire of your heart and make all your plans succeed” (Psalm 20:4). God have given me success in completing this thesis. I hope that God also give all of people who helped me success in their life.

Ruth Widyasari

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(14)

xiii

Page

TITLE PAGE………i

APPROVAL PAGES………..………ii

STATEMENT OF WORK’S ORIGINALITY……...………v

LEMBAR PERNYATAAN PERSETUJUAN PUBLIKASI………....vi

ABSTRACT………...………vii

ABSTRAK………ix

ACKNOWLEDGEMENTS………....xi

TABLE OF CONTENTS………..xiii

LIST OF TABLES………....xvi

LIST OF FIGURES……….…xvii

LIST OF APPENDICES………...xviii

CHAPTER I. INTRODUCTION A. Research Background... 1

B. Problem Formulations ... 4

C. Problem Limitation ... 4

D. Research Objectives ... 5

E. Research Benefits ... 5

F. Definition of Terms ... 6

CHAPTER II. REVIEW OF RELATED LITERATURE A. Theoretical Description ... 8

(15)

xiv

2. Test ... 10

3. Design and Test Development ... 15

B. Theoretical Framework ... 19

CHAPTER III. METHODOLOGY A. Research Method ... 22

B. Research Participants ... 28

C. Research Instruments ... 29

D. Data Gathering Technique ... 30

E. Data Analysis Technique ... 30

F. Research Procedure ... 31

CHAPTER IV. RESEARCH RESULTS AND DISCUSSION A. How the Achievement Multiple-Choice Item Narrative Reading Test is Designed………..….35

1. Research and Information Collecting ………35

2. Planning………..39

3. Developing Preliminary Form of Product………….. ……41

4. Preliminary Field Testing………...……42

5. Main Product Revising………...50

B. What the Achievement Multiple-Choice Item narrative Reading Test Looks Like ……..………..52

CHAPTER V. CONCLUSIONS AND SUGGESTIONS

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(16)

xiv

A. Conclusions ... 64

B. Suggestions ... 66

REFERENCES….………..68

APPENDICES………70

(17)

xiv

Page Table 4.1 Design Statement of the Test……….37 Table 4.2 Blueprint of the Test………..39 Table 4.3 The Results of Item Facility (IF) and Item Discrimination (ID)...42 Table 4.4 The Test Revision………..51

LIST OF FIGURES

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(18)

xiv

Page Figure 1.1 Test, Achievement, and Teaching………1 Figure 2.1 The Formula of Final Score………...15 Figure 2.2 Test Development………..18 Figure 3.1 The Combination of R & D and Bachman’s and Palmer’s Test

Development……….23

(19)

xiv

Page

Appendix 1 The Interview……….…71

Appendix 2 The Lesson Plan……….73

Appendix 3 The Syllabus ……….…….82

Appendix 4 The Texts………..…..92

Appendix 5 The Answer Sheet………..99

Appendix 6 The Questionnaire………100

Appendix 8 The Result of Counting IF and ID………101

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(20)

1 CHAPTER I INTRODUCTION

In Chapter I, the writer elaborates the research background, problem formulations, problem limitation, research objectives, research benefits, and definition of terms.

A. Research Background

English is one of the most important subjects in senior high school in Indonesia since English is one of the subjects, which is tested in National Examination. English plays important role in National Examination because it determines the students’ graduation. Because of this reason, the process of teaching and learning English is also essential. In senior high school, English is taught from the tenth grade up to the twelfth grade. In the process of teaching English, there are also assessing and testing process to monitor the students’ mastery in English. The relationship among teaching, assessment, and test can be seen in figure 1.1 below:

Figure 1.1: Test, achievement, and teaching (Brown, 2004: 5) Teaching

Assessment Tests

(21)

Figure 1.1 shows that teaching process cannot be separated from assessment and tests. Tests are a part of assessment and teaching, while assessment is also a part of teaching.

In this research, the writer is challenged to design a test since tests are very important in teaching and learning process. Tests are very useful to record the students’ achievement and as tools for teacher to develop teaching and learning process. Brown (2004: 4) also states that

Tests are prepared administrative procedures that occur at identifiable times in a curriculum when learners muster all their faculties to offer peak performance, knowing that their responses are being measured and evaluated.

In this research, the writer focuses on designing a multiple-choice item reading test. Reading as one of language skills in learning English is chosen to design the test because reading is essential skill for success in all education contexts Brown (2004:185). Besides, in National Examination, most of the questions test the reading skill.

There are some techniques of language tests that can be applied in schools such as short-answer tests, fill-in-the-blanks tests, and multiple-choice item tests. In this research, the writer designs a multiple-choice item reading test as an achievement test. According to Hughes in Brown (2004: 55), multiple-choice items, which may appear to be the simplest kind of item to construct, are extremely difficult to design correctly. Therefore, it will be challenging the writer to design a multiple-choice item reading test correctly. In addition, an achievement test is related directly to classroom lessons, units, or even a total curriculum (Brown, 2004:47). In this research, the writer will design a

multiple-PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(22)

choice test for achievement test, because the writer wants to monitor the students’ understanding after they had been taught a certain topic.

As stated above that the writer focuses on designing a multiple-choice reading test, therefore the writer also considers the reading performance of the students. There are four types of reading performances. They are perceptive, selective, interactive, extensive (Brown, 2004: 189). In this thesis, the writer focuses on measuring students’ performance in interactive reading.

…interactive reading types are stretches of language of several paragraphs to one page or more in which the reader must, in psycholinguistics sense, interact with the text. Typical genres that lend themselves to interactive reading are anecdotes, shorts narratives and descriptions. The focus on an interactive task is to identify relevant features (lexical, symbolic, grammatical, and discourse) within texts of moderately short length with the objective of retaining the information that is processed (Brown, 2004: 189).

In addition, several reading genres are taught in schools among others descriptive, narrative, recount, and report. To limit the scope of the test the writer chose narrative. Narrative is chosen because narrative is a classroom lesson, which is taught in the second semester of the tenth grade of senior high school as stated in the syllabus. Furthermore, one interesting feature of narratives texts in particular is that they appear to induce visualization in the reader as part of reading process-readers report ‘seeing’ scenes in their head when they read such texts (Alderson, 2000: 64).

The test is tested to the respondents from the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School because in the tenth grade all students have the same topic. The students have not been divided into some departments such as IA (Ilmu Alam), IS (Ilmu Sosial), or Bahasa. This study aims to design a

(23)

multiple-choice item test and investigate the validity of the test. The writer will only investigate the validity of the test because it is the quality that provides the major justification for using test scores or number as a basis for making inferences or decisions. In addition, evaluating the overall usefulness of a given test is essentially subjective, since this involves value judgments on the part of the test developer (Bachman and Palmer, 1996:19).

This study is very challenging since it will relate to the curriculum and syllabus, the participation of students in Pangudi Luhur St. Yosef Surakarta Senior High School, the English teacher of the tenth grade students in Pangudi Luhur St. Yosef Senior High School and the test itself.

B. Problem Formulations

In this research, the writer proposed the following questions:

1. How is the achievement multiple-choice item narrative reading test designed?

2. What does the achievement multiple-choice item narrative reading test look like?

C. Problem Limitation

In this study, the writer focuses on designing a multiple-choice items reading test for the tenth grade students of Pangudi Luhur St. Yosef Surakarta Senior High School, and investigating the validity of the test included face validity, content validity, and construct validity.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(24)

D. Research Objectives

There are two objectives of designing multiple-choice reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School:

1. To describe how the achievement multiple-choice item narrative reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School is designed;

2. To present the achievement multiple-choice item narrative reading test for the tenth grade students of Pangudi Luhur St. Yosef Senior High School.

E. Research Benefits

There are three benefits of this research, as follows:

1. This study contributes to the English teacher of the tenth grade of Pangudi Luhur St. Yosef Senior High School. The teacher will have an alternative to test the students’ reading ability in narrative by using multiple-choice reading test since multiple-choice tests are infrequently used as achievement tests for classroom lesson.

2. This study benefits the writer to answer the research problems and also as a teacher candidate, the writer needs to know how to make the multiple-choice item reading test and measure the quality of the multiple-multiple-choice item reading test.

3. Hopefully, this study will give contribution to other researchers who want to conduct researches related to designing multiple-choice tests and measuring the quality of multiple-choice item reading test.

(25)

F. Definition of Terms

In this study, there are four terms which are often used by the writer. To clarify the meaning of the terms, the writer provides a list below:

1. Design in general is creating a new set of materials that fix the learning objectives and specific subject area of particular learners (Hutchinson and Waters, 1994:106). While design in test development is the process of describing the purpose (s) of the test, identifying and describing tasks in the TLU (Target Language Use: a context in which the test takers will be using the language outside the test itself) domain, describing the characteristics of the language users/test takers, defining the construct to be measured, developing a plan for evaluating the qualities of usefulness, identifying resources and developing a plan for their allocation and management (Bachman and Palmer, 1996: 86-89).

2. Reading is transaction between the reader and the text in which the reader’s interpretation reflects both the meaning intended by the author and the meaning constructed by the reader (Armbruster & Osborn, 2002: 7). Reading for general comprehension is the ability to understand information in a text and interpret it appropriately (Grabe & Stoller, 2002: 9).

3. Test is a method of measuring a person’s ability, knowledge, or performance in a given domain (Brown, 2004: 3). The domain is overall proficiency in a language – general competence in all skills of a language (Brown, 2004: 4). In this study, the given domain is reading comprehension of narrative texts including obtaining main idea (topic), expressions/idioms/phrases in context,

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(26)

inference (implied detail), grammatical features, detail, excluding facts not written, supporting idea(s), and vocabulary in context.

4. Multiple-choice tests are the tests that have a stem, which presents a stimulus, and several (usually between three and five) options or alternatives to choose from (Brown, 2004: 56).

(27)

8 CHAPTER II

REVIEW OF RELATED LITERATURE

In this chapter, the writer presents theoretical description from the experts, which are useful to conduct the research and theoretical framework of the study. The theoretical description contains the theoretical description of reading, test, and design and test development. While in theoretical framework, the writer summarizes and synthesizes the theories, which are useful to conduct the research and solve the research problems.

A. Theoretical Description

The writer uses some theories as a basis in conducting this research. They are the theoretical description of reading, test, and design and test development. The theoretical description of reading is divided into definition of reading, types of reading and reading tasks. The theoretical description for the test is divided into the definition of the test, the purpose of the test, the format of the test, test validity, marking and scoring.

1. Reading

a. Definition of Reading

Reading is transaction between the reader and the text in which the reader’s interpretation reflects both the meaning intended by the author and the meaning constructed by the reader (Armbruster & Osborn, 2002: 7). Reading for

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(28)

general comprehension is the ability to understand information in a text and interpret it appropriately (Grabe & Stoller, 2002: 9).

b. Types of Reading Performance

For considering assessment procedures, several types of reading performance are typically identified, and these will serve as organizers of various assessment tasks as follows: perceptive, selective, interactive, extensive (Brown, 2004: 189).

In designing the test, the writer focuses on interactive reading. According to Brown (2004: 189) “Included among interactive reading types are stretches of language of several paragraphs to one page or more in which the reader must, in psycholinguistics sense, interact with the text. That is, reading is a process of negotiating meaning; the reader brings to the text a set of schemata for understanding it, and intake is the product of that interaction. Typical genres that lend themselves to interactive reading are anecdotes, shorts narratives and descriptions. The focus on an interactive task is to identify relevant features (lexical, symbolic, grammatical, and discourse) within texts of moderately short length with the objective of retaining the information that is processed”. The writer designs a multiple-choice item reading test, which focuses on a certain genres that is narrative because it is in accord with the curriculum for grade X. c. Reading Tasks

The reading tasks that the writer used to design the test depend on the types of reading performance. The types of reading performance, which the writer focuses on, are interactive reading. Interactive reading can be implemented in

(29)

various tasks, in particular: cloze tasks, impromptu reading plus comprehension questions, short-answer tasks, editing (longer texts), scanning, ordering tasks, information transfer: reading charts, maps, graphs, diagram (Brown, 2004:201-210). In designing a multiple-choice item reading test, which focuses on narrative, the writer designed two kinds of tasks: cloze tasks and impromptu reading plus comprehension questions.

According to Brown (2004: 201-206), cloze task is one of the most popular types of reading assessments task. In the written language, a sentence with a word left out should have enough contexts that a reader can close that gap with a calculated guess. While impromptu reading plus comprehension questions technique is undoubtedly the oldest and the most common. This technique covers the comprehension of these features: main idea (topic), expressions/idioms/phrases in context, inference (implied detail), grammatical features, detail, excluding facts not written, supporting idea(s), and vocabulary in context.

2. Test

a. Definition of Test

Test is a method of measuring a person’s ability, knowledge, or performance in a given domain. It is an instrument- a set of techniques, procedures, or items- that requires performance in the part of the test-takers. (Brown, 2004:3).

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(30)

b. The Purpose of The Test

According to Brown (2004:43), the first task teacher will face in designing a test for the students is to determine the purpose of the test. Defining the purpose of the test will help teacher choose the right kind of test, and it also will help the teacher to focus on the specific objectives of the test. Based on the objectives, tests are divided into five types, in particular language aptitude tests, language proficiency tests, placement tests, diagnostic tests, and achievement tests.

In this study, the writer focuses on achievement test. Brown (2004:47) stated that an achievement test is related directly to classroom lesson, units, or even a total curriculum. Achievement tests are (or should be) limited to particular material addressed in a curriculum within a particular time frame and are offered after a course has focused on the objectives in question.

c. The Format of The Test

The writer develops an achievement test by designing a task so-called multiple-choice test, which is also known as selection items. Brown (2005: 43) states a multiple-choice format is basically receptive mode (students read and select, but produce nothing). According to Brown (2004: 55), multiple choice items, which may appear to be the simplest kind of item to construct, are extremely difficult to design correctly. Here are the characteristics of multiple choice items according to Brown (2004: 56):

1. Multiple-choice items are all receptive, or selective, response items in that the test-taker chooses from a set of responses rather than creating a response.

(31)

2. Every multiple choice item has a stem, which presents a stimulus, and several (usually between three and five) options or alternatives to choose from.

3. One of those options, the key, is the correct response, while the others serve as distractors.

d. Test Validity

The writer also investigates the validity of the test. Validity is the extent to which inferences made from assessment result are appropriate, meaningful, and useful in terms of the purpose of the assessment (Brown, 2004: 22). There are three criteria of validity, which are investigated by the writer, as follow: face validity, content validity, and construct validity.

1. Face validity refers to the test’s surface credibility or public acceptability (Alderson, Clapham, and Wall, 1995: 172). Essentially face validity involves an intuitive judgment about test’s content by people whose judgment is not necessarily ‘expert’. (Alderson, Clapham, and Wall, 1995: 172)

2. Content validity is the representativeness or sampling adequacy of the content-the substance, content-the matter, content-the topics- of a measuring instrument. Content validation involves gathering the judgment of ‘expert’. (Alderson, Clapham, and Wall, 1995: 173)

3. Construct validity is therefore used to refer to the extent to which we can interpret a given test score as an indicator of the abilities, or construct, we want to measure. Construct validation is the on-going process of demonstrating that a particular interpretation of test scores is justified, and involves, essentially, building a logical case in support of a particular

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(32)

interpretation and providing evidence justifying that interpretation ( Bachman and Palmer, 1997:21-22).

In addition, in this study, the writer measure construct validity through measuring item facility (or item difficulty), item discrimination (differentiation), and distractor analysis by using Microsoft Excel. Excel is an electronic spreadsheet program that can be used for storing and organizing data (French, 2011:1).

1. Item facility (IF) is a statistic used to examine the percentage of students who correctly answer a given item (Brown, 2005:66). IF refers to the proportion of students who answered the question correctly, we calculated the IF using COUNTIF function for the letter corresponding to the correct answer (Elvin, 2003). The COUNTIF function is used to count up the number of cells in a selected range that meet certain criteria (French, 2011:1). Appropriate test items will generally have IFs that range between 0,15 and 0,85 (Brown, 2004: 59)

2. Item discrimination (ID) is a statistic that indicates the degree to which an item separates the students who performed well from those who did poorly on the test as a whole. These groups are sometimes referred to as the “high” and “low” scorers or “upper” and “lower” proficiency students (Brown, 2004:68). ID is usually the difference between the IF for the top third of test takers and the IF for the bottom third of test takers for each item on a test, we can calculated the ID using SUM function (Elvin, 2003). The SUM function will add together the contents of all the cells in the range (French, 2011:1). High

(33)

discriminating power would approach a perfect 1,0 (one) and no discriminating power at all would be 0 (zero).

3. According to Brown (2004:60), distractor efficiency is the extent to which (a) the distractor “lure” a sufficient number of test-takers, especially lower-ability ones, and (b) those responses are somewhat evenly distributed across all distractors. In addition, Elvin (2003) stated that the quality of a test item may be poor. To try to identify these potential sources of measurement error, I calculate the ratio of students answering each question to students taking the test. It can be calculated by using SUM function.

e. Marking and Scoring

In designing the test, the writer also determines how to give mark and score. There are basically two types of marking: subjective marking which is usually used for marking test of writing and speaking, and objective marking which is used for multiple-choice, true/false, error recognition and other item types where the candidate is required to produce a response which can be marked as either ‘correct’ or ‘incorrect’ (Alderson, Clapham, and Wall, 1995:106). Thus, in this research, the writer uses objective marking.

If the test consists of a number of objective subsets (for example: multiple-choice), then each item may have been assigned a mark of 1 (one) if correct and 0 (zero) if wrong (Alderson, Clapham, and Wall, 1995:148). In this research, the writer gives score based on the mark of correct number which is divided by two (2). The formula is as follow:

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(34)

Figure 2.1: the Formula of Final Score

3. Design and Test Development

Design is in general a linear process, but in some cases some activities are iterative, that is, will need to be repeated a number of times (Bachman and Palmer, 1996: 86).

In addition, design is arranging materials into a fixed and good material. The designing is the same as creating a new set of materials that fix the learning objectives and specific subject area of particular learners (Hutchinson & Waters, 1994:106). To design the multiple-choice reading test, the writer will apply some steps of test development.

Test Development is the entire process of creating and using test, beginning with its initial conceptualization and design, culminating in one or more archived tests and the results of their use (Bachman & Palmer, 1996: 85). According to Bachman and Palmer (1996: 86-91), there are three stages of test development:

1. Stage One: Design

The product of the design stage is a design statement, which is a document that includes the following components:

a) a description of the purpose (s) of the test,

#

2

(35)

b) a description of the Target Language Use (TLU) domain and task types: Target Language Use is context in which the test takers will be using the language outside of the test itself. There are two types of TLU, real-life domain, in which language is used essentially for purposes of communication. The other type of domain consists of situations in which language is used for the purpose of teaching and learning of language, which is called as language instruction domain. While task types is in the form of multiple choice test.

c) a description of the test takers for whom the test intended, d) a definition of the construct (s) to be measured,

e) a plan for evaluating the qualities of usefulness, and

f) an inventory of required and available resources and a plan for their allocation and management.

2. Stage two: Operationalization

Operationalization involves developing test task specifications for the types of test tasks to be included in the test, and a blueprint that describes how test tasks will be organized to form actual tests. Operationalization also involves developing and writing actual test tasks, writing instructions, and specifying the procedures for scoring the test. By specifying the conditions under which language use will be elicited and the method for scoring responses to these tasks, we are providing the operational definition of the construct.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(36)

In addition, Brown (2004: 55) explains some practical steps to design multiple-choice test items: design each item to measure a specific objective, state both stem and options as simply and directly as possible, make certain that the intended answer is clearly and the only correct one, use item indices to accept, discard, or revise items.

3. Stage three: Test administration

The test administration stage of the test development involves giving the test to a group of individuals, collecting information, and analyzing this information for two purposes:

a) assessing the usefulness of the test, and

b) making the inferences or decisions for which the test intended.

The Summary of three stages of Bachman’s and Palmer’s Test Development could be seen in the figure 2.2 below:

(37)

Test Development

Stages / activities Products

Figure 2.2: Test Development (Bachman and Palmer, 1996: 86) 1. Design

Describing Identifying Selecting Defining Developing Allocating Managing 2. Operationalization Selecting Specifying Writing Consideration of qualities of usefulness 3. Administration Administering Collecting feedback Analyzing Archiving Design statement

Purpose of the test Description of the TLU

domain and task types Characteristics of test takers Definition of the construct (s) Definition for evaluating

the qualities of usefulness Inventory of available

resources and plan for their allocation and management

Blueprint

Test structure Number of parts/tasks Salience of parts Sequence of parts Relative importance of

parts/tasks

Number of tasks per part Test task specifications Purpose

Definition of construct (s) Setting

Time allotment Instructions

Characteristics of input and expected response

Scoring method

Test 1 Feedback on Usefulness Qualitative Quantitative Test n Test 2

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(38)

B. Theoretical Framework

In this section, the writer summarized and synthesized the theory above with the study. There were three theoretical descriptions in this chapter covered reading, test, and design and test development. The theoretical descriptions above were needed to give the writer a basis to do the research.

By doing the research, the writer wanted to measure students’ ability in reading. To measure the students’ ability in reading, the writer designed a test. The format of the test is multiple-choice test. Although it seemed the easiest way to make the test, the writer needed to consider some elements to design the tests. Moreover, in this study the writer did not merely design the test but also consider the usefulness or quality of the test. Considering the quality of the test, the writer needed to try-out the test and investigate the ID, IF, and distractor efficiency.

In this study, the writer focused on a reading genre so-called narrative. To design the multiple-choice reading test, the writer focused on interactive reading types. After selecting the types of reading performance, the writer decided the tasks. There were two kinds of reading tasks of the test, which the writer had designed. There were cloze tasks and impromptu reading plus comprehension questions. Both tasks were chosen because they could explore the students’ ability in understanding texts.

In designing the test, the writer used the guideline of Bachman and Palmer (1996: 86-91) as stated previously. There were three stages done by the writer to answer the research problems:

(39)

1. Design

In this stage the writer described, identified, selected, defined, developed, allocated, and managed the test. This stage was a preparation before writing the test. In this step, the writer produced a design statement. A design statement was needed to limit the scope of the test and to give a guideline to produce the test. A design statement contained the purpose of the test, description of the TLU (Target Language Use) domain and task types, characteristics of the test takers, definition of the construct (s), definition for evaluating the qualities of usefulness, inventory of available resources and plan for their allocation and management. To do this stage, the writer would interview the English teacher and do the library study to get the syllabus and resources.

2. Operationalization

Operationalization was a stage when the writer was writing the test. This stage covered selecting, specifying, and writing the test. The product of this stage was a blueprint and the test. A blueprint contained a test structure and test task specifications. A test structure contained number of parts/tasks, salience of parts, sequence of parts, relative importance of parts/tasks, and number of tasks per part. Test task specifications contained purpose of the test, definition of construct (s), setting, time allotment, instructions, characteristics of input and expected response, and scoring method. Since the designed test was a multiple-choice test, the writer also designed the multiple-choices (items). The writer applied some Brown’s practical steps to design test items (Brown, 2004: 55): design

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(40)

each item to measure a specific objective, state both stem and options as simply and directly as possible, make certain that the intended answer is clearly and the only correct one, use item indices to accept, discard, or revise items. After finishing this stage, the test was ready to be tested.

3. Administration

The last stage of test development was administration, which covered administering, collecting feedback, analyzing, archiving. This stage, as stated above, involved giving the test to the test takers, collecting information, and analyzing the information. The writer would give the test to the tenth grade students of Pangudi Luhur St. Yosef Senior High School. After giving the test, the writer would collect information related to the test. The writer would get the information related to the tests from both the students and the English teacher. From the students, the writer would get the information in a form of answer sheets and from the English teacher, the writer would get the answer of the questionnaire. Then the writer would analyze the information.

The research did not only involve the test development, but it would apply a specific research methodology so-called Research and Development (R & D). Therefore, the writer would combine the test development and R & D Cycle in Chapter III.

(41)

22 CHAPTER III

METHODOLOGY

In this research, the writer elaborates the methodology used for designing a set of multiple-choice item reading test for the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year. The methodology covers research method, research participants, research instruments, data gathering techniques, data analysis technique and research procedure.

A. Research Method

The method used in this research was Research and Development (R & D) because the writer designed multiple-choice reading test. Educational R & D is a process used to develop and validate educational products (Borg & Gall, 1983: 772). According to Borg, there are ten major steps in R & D: research and information collecting, planning, developing preliminary form of product, preliminary field testing, main product revision, main field testing, operational product revision, operational field testing, final product revision, dissemination and implementation.

In doing this research, the writer only used five steps of R & D process because by doing those five steps the research problems had been answered. The five steps which were used by the writer are research and information collecting, planning, developing preliminary form of product, preliminary field testing, and main product revising.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(42)

Moreover, in doing this research the writer combined R & D with Bachman’s and Palmer’s test development. The Combination of R & D and Bachman’s & Palmer’s test development can be seen in the figure below.

Stage 1:

Stage 2:

Stage 3:

Stage 4:

Stage 5:

Figure 3.1: the Combination of R & D and Bachman’s and Palmer’s Test Development

a. Research & Information

Collecting

b. Design: produce design

statement

Preliminary Field Testing

Operationalization: developing and writing actual test tasks, writing instructions, specifying the

procedures for scoring Developing

Preliminary Form of Product

b. Make the blueprint a. Planning =

Operationalization: selecting and specifying the materials

Administration

Main Product Revising

Through literature review and interview

(43)

Figure 3.1 shows that in this research, the writer applied five steps of R & D so-called research and information collecting, planning, developing preliminary form of product, preliminary field testing and main product revising. Then the writer combined R & D and the test development, which consisted of three stages, in particular: design, operationalization, and test administration. The first stage that is design, was done after the first step of R& D that is research and information collecting. In this stage, the writer made a guideline so-called design statement before producing a multiple-choice reading test. The writer would limit the scope of the test by making design statement of the test. In the second stage of Bachman’s and Palmer’s Test Development, operationalization was divided into two steps of R & D, step two and three of R & D cycle, planning and developing preliminary form of product. As stated in the previous chapter that operationalization consisted of three activities: selecting, specifying, and writing. The activities of selecting and specifying the test would be done in step two of R & D cycle, while writing the test would be done in step three, developing preliminary form of product. The last one was test administration was done in step four of R & D cycle, preliminary field testing. After gathering the answer sheets, the writer analyzed the result of the test and if needed, reported to the teacher. Then, the writer revised the test’s item if needed.

1. Research and Information Collecting

In the first step, research and information collecting, the writer collected the literature review. Literature review is undertaken to collect research findings and

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(44)

other information pertinent to the planned development (Borg & Gall, 1983: 777). To obtain the literature review, the writer conducted an interview and a library study. The writer interviewed the teacher who taught the students of grade X of Pangudi Luhur St. Yosef Surakarta Senior High School. By doing the interview, the writer obtained the data on the tests specification. The writer needed the test specification because “test writer needs guidance on practical matters that will assist test construction.” (Alderson, Clapham, and Wall, 1995:11). The writer had conducted library study to obtain some theories related to the study as well as the syllabus.

After finishing this step, the writer applied Bachman’s and Palmer’s test development stage one, design. By doing the stage one, design, the writer would produce design statement which covered purpose of the test, description of TLU (Target Language Use) domain and task types, characteristics of the test takers, definition of the construct (s), definition for evaluating the qualities of usefulness, inventory of available resources and plan for their allocation and management.

In research and information collecting, the writer did interview to find out characteristics of the test takers. The participant of the interview was the English teacher who taught the tenth grade of Pangudi Luhur Senior High School. The writer also conducted library study to obtain some aspects of design statement such as purpose of the test, description of TLU (Target Language Use) domain and task types, definition of the construct (s), definition for evaluating the qualities of usefulness, inventory of available resources and plan for their allocation and management. In conducting the library study, the writer needed to

(45)

know some theories about reading, test, design and test development. Moreover, the writer needed to know the school’s curriculum and syllabus to enable the writer to define the design statement. In this step, the writer had produced the overall design statement, which would be used as a guideline to produce the test.

2. Planning

In the second step, planning, Borg and Gall (1983: 781) stated that a good plan can help the developer avoid much wasted work during later phases of good R & D cycle.

The writer did the planning by doing operationalization of Bachman’s and Palmer’s Test Development, which was included selecting and specifying the materials to construct the test. By doing selecting and specifying the materials the writer would be able to make a blueprint of the test. A blueprint contained a test structure and test task specifications. A test structure contained number of parts/tasks, salience of parts, sequence of parts, relative importance of parts/tasks, and number of tasks per part. Test task specifications contained purpose of the test, definition of construct (s), setting, time allotment, instructions, characteristics of input and expected response, and scoring method. This step would be done by conducting library study.

3. Developing Preliminary Form of Product

Then the writer did the third step which was developing preliminary form of product.” Developing preliminary form of product includes preparation of

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(46)

instructional materials handbooks, and evaluation devices.” (Borg & Gall, 1983: 775)

In this fourth step, the writer applied the last activity of operationalization which was writing the test. Bachman and Palmer states that operationalization involves developing and writing actual test tasks, writing instructions, and specifying the procedures for scoring the test. Therefore, in this step the writer made the test, answers, and scoring system.

4. Preliminary Field testing

The writer conducted preliminary field testing by doing a stage of administration (Bachman’s and Palmer’s Test Development). Administration consisted of four activities: administering, collecting feedback, analyzing, archiving. The participants of this step are the tenth grade students of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year as well as the English teacher who taught the tenth grade students of Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year. Administering activity would involve conducting the test to the students of the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year as the test-takers and distributing the questionnaire to the English teacher. After that, the writer collected the students’ answer sheets and feedback from the English teacher as the data, this activity was involved in collecting feedback. From the data gathered, the writer would analyze the students’ answer sheet to investigate construct validity of the test through counting IF, ID, and distractor efficiency of

(47)

the test’s items. Furthermore, the writer also analyzed the feedback from the questionnaire to investigate face validity and content validity.

5. Main Product Revising

The last one, main product revision, the writer revised the test based on investigation of validity of the test. From the result of the validity investigation, the writer was able to make the revision because the results of validity investigation would show the quality of the test and decide which one should be revised and which one should not be revised.

B. Research Participants

The participants of this research were:

1. Participant of Research and Information Collecting

The participant of research and information collecting was the teacher of the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year. In doing the first step, research and information collecting, the writer conducted an interview to the teacher as the participant. 2. Participants of Preliminary Field Testing

The participants of preliminary field testing were the tenth grade students of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year and also the English teacher of grade ten or the expert. The number of the students who joined the test were 236 students from seven classes. They were in the second semester of the tenth grade and they had learned the material about narrative. The participants were asked to answer the

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(48)

test. Then their answer sheets were collected and analyzed. Then, the writer also distributed questionnaires to the English teacher of Pangudi Luhur St. Yosef Surakarta Senior High School. The questionnaires were used to investigate face validity and content validity of the test.

C. Research Instruments

There were two instruments used in this research: interview and questionnaires

1. Instrument of Research and Information Collecting

Interview was used as the instrument to do research and information collecting. The interview was done to obtain the data of the test specification. The writer interviewed the English teacher who taught the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year.

2. Instrument of Developing Preliminary Field testing

The instrument of developing preliminary field testing was questionnaires. The questionnaires were distributed to the English teacher or expert to obtain the data of face validity and content validity.

D. Data Gathering Technique

To gather the data, the researcher conducted an interview with the English teacher of the tenth grade Pangudi Luhur St. Yosef Surakarta Senior High School

(49)

in 2009/2010 academic year, library study, and distributed questionnaire to the English teacher as the expert to obtain the feedback.

1. Data Gathering Technique of Research and Information Collecting: a. Interview was used to gather the data on research and information

collecting. The writer interviewed the English teacher who taught the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School to obtain the data on the test specifications.

b. Library Study was used to obtain the syllabus and lesson plan of the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School. Library study was conducted to gather the data on research and information collecting.

2. Data Gathering Preliminary Field Testing:

Questionnaire was used to obtain the data of face validity and content validity of the test. The writer distributed the questionnaires to the English teacher who taught the tenth grade Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year.

E. Data Analysis Technique

The data based on the interview was analyzed by using descriptive analysis. Since the interview resulted on the test specifications, the descriptive analysis was used to design the test. This data analysis technique was done in step one, research and information collecting.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(50)

The data gathered from questionnaires were used to obtain the feedback from English teacher who taught the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year. These data were used for investigating face validity and content validity of the test. The data were analyzed by using descriptive analysis.

F. Research Procedure

In this research, the writer designs a set of multiple-choice reading test as an achievement test and investigates the validity of the multiple-choice reading tests, which are tested in the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School. The whole process consists of five steps:

1. Research and Information Collecting

In the first step, research and information collecting involved stage one of Bachman’s and Palmer’s Test Development, that is design by doing library study and interview. The writer conducted an interview to the participant who was the English teacher who taught the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year. The writer also conducted the library study to obtain some theories of reading, test, design and test development, and the syllabus and lesson plan of the tenth grade Pangudi Luhur St. Yosef Surakarta Senior High School.

By doing stage one, design, the writer would produce design statement, which covered purpose of the test, description of TLU (Target Language Use) domain and task types, characteristics of the test takers, definition of the construct

(51)

(s), definition for evaluating the qualities of usefulness, inventory of available resources and plan for their allocation and management.

In research and information collecting, the writer would do interview to find out characteristics of the test takers and library study to obtain some aspects of design statement. They are purpose of the test, description of TLU (Target Language Use) domain and task types, definition of the construct (s), definition for evaluating the qualities of usefulness, inventory of available resources and plan for their allocation and management.

2. Planning

In the second step, planning, Borg and Gall (1983: 781) stated that a good plan can help the developer avoid much wasted work during later phases of good R & D cycle. The writer did the planning by doing operationalization, which was included selecting and specifying the materials by conducting library study. After conducting the selecting and specifying the materials, the writer would be able to make a blueprint of the test.

3. Developing Preliminary Form of Product

Then the writer did the third step which was developing preliminary form of product.” Developing preliminary form of product includes preparation of instructional materials handbooks, and evaluation devices.” (Borg & Gall, 1983: 775)

In this fourth step, the writer applied the last activity of operationalization, which was writing the test. Bachman and Palmer states that operationalization involves developing and writing actual test tasks, writing instructions, and

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(52)

specifying the procedures for scoring the test. Therefore, in this step the writer made the test, answers, and scoring system.

4. Preliminary Field testing

The writer conducted preliminary field testing by doing a stage of administration (Bachman’s and Palmer’s Test Development). Administration consisted of four activities: administering, collecting feedback, analyzing, archiving. In administering, the writer needed participants to solve the research problem. The participants are the students of tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year as the test-takers and the English teacher of tenth grade students of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year to whom the questionnaire given.

After that, the writer collected the students’ answer sheets and feedback from the English teacher as the data, this activity was involved in collecting feedback. From the data gathered, the writer would analyze the students’ answer sheet to investigate construct validity of the test through counting IF, ID, and distractor efficiency of the test’s items. Furthermore, the writer also analyzed the feedback from the questionnaire to investigate face validity and content validity. The data then would be interpreted into descriptive analysis.

5. Main Product Revising

Main product revision would be done if it was needed. The revision was made based on the result of validity investigation of the test, which obtained from the analysis of the students’ answer sheets and the questionnaire.

(53)

Finally, by doing the five process of R & D cycle, the research problems could be answered. The data obtained from doing the research procedure would be called as research result. The writer would report and discuss the research results in Chapter IV.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(54)

35 CHAPTER IV

RESEARCH RESULTS AND DISCUSSION

In this chapter, the writer presents research finding and elaborates the research finding into a discussion.

In this research, the writer applied Research and Development cycle (R & D). The writer only used five steps of R & D process because by doing those five steps the research problems has been answered. The five steps which were used by the writer are research and information collecting, planning, developing preliminary form of product, preliminary field testing, and main product revising.

A. How the Achievement Multiple-Choice Item Narrative Reading Test is Designed

1. Research and Information Collecting

In this first step, there were two main activities. They were interview and literature review.

a. Interview

The writer interviewed the teacher who taught the students of grade X of Pangudi Luhur St. Yosef Surakarta Senior High School as a participant on Tuesday, 20th April 2010. By doing the interview, the writer obtained the data on the characteristics of the test takers (see the design statement on page 36-37).

The writer proposed some questions (see the appendix 1 page 53) related to the students’ characteristics. The teacher explained that some students were

(55)

outstanding students while most of students were not good enough in English. Some students who were outstanding students were very fluent in speaking English but they were not good enough in understanding reading text.

From the interview, the writer could draw some conclusions related to the test specification. The writer needed the test specification because “test writer needs guidance on practical matters that will assist test construction.” (Alderson, Clapham, and Wall, 1995:11).

b. Literature Review

In the first step, the writer had conducted literature review to obtain some theories related to the study and the syllabus and lesson plan of the tenth grade of Pangudi Luhur St. Yosef Surakarta Senior High School so that the writer had better understanding before designing the test. The writer conducted literature review on Tuesday, 20th April 2010. From the syllabus and lesson plan, the writer obtained some data related to the test specification and teaching narrative itself.

After conducting an interview and a literature review, the writer found the research results. The research result of this first step was a design statement. A design statement is a document that includes the following components a description of the purpose (s) of the test, a description of the Target Language Use (TLU) domain and task types, a description of the test takers for whom the test intended, a definition of the construct (s) to be measured, a plan for evaluating the qualities of usefulness, and an inventory of required and available resources and a plan for their allocation and management Bachman and Palmer (1996: 86).

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(56)

Design Statement

Purpose of the test

The test was designed as an achievement test which aimed to monitor the students’ mastery on narrative. Achievement tests were (or should be) limited to particular material addressed in a curriculum within a particular time frame and are offered after a course has focused on the objectives in question. (Brown, 2004: 47) By doing the test, the students were expected to achieve some indicators stated in the syllabus, as follows:

a. Students are able to identify the meaning of certain words in the text;

b. Students are able to identify the meaning of certain sentence in the text;

c. Students are able to identify the rhetorical steps of the narrative.

Description of the Target Language Use

(TLU) domain and task types

a. TLU: Language Instructional b. Task Types: multiple-choice test

Characteristics of test takers

Personal characteristics:

a. Age: They are 15-16 years old b. Sex : male and female

b. Native Language: Javanese

c. Prior experience with a given test: they were taught narrative more than three meetings.

Level of language knowledge:

a. A literal understanding of a text: for most of students b.An understanding of meaning that are not directly

stated in the text: advanced students

Definition of construct (s)

Syllabus-based construct: since the test was designed as an achievement test, it was designed to achieve the objectives of the lesson.

(57)

Design Statement

Plan for evaluating the qualities of

usefulness

Validity: face validity, content validity, and construct validity

Inventory of required and available resources and a plan for their

allocation and management

Resources:

a.Theory of reading b.Theory of test

c.Theory of designing multiple choice test d.Theory of scoring and marking system Allocation and management:

Each resource contained a text ranging from 4 up to 8 paragraphs. The writer simplified some texts without omitting the essential events. Each text had five (5) questions.

Table 4.1: Design Statement of the Test

From the design statements, the writer concluded that the writer had to design a test to achieve the objectives of the lesson. To achieve the objectives of the test, there are some indicators which would be a guideline for the writer in writing the test. The indicators appeared in the questions of the test. Considering the reading ability of the students, the writer wrote a test which the advanced and less advanced students understand the instructions, the questions, and the items.

To evaluate the qualities of the test, the writer had to prepare some instruments to evaluate the validity of the test. To evaluate face validity and content validity, the writer had to prepare questionnaire. While to evaluate construct validity, the writer investigated the item facility, item discrimination, and distractor efficiency.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(58)

From the test specification and the result of interview and library study, the writer had some guidelines to do step two, planning.

2. Planning

Step two, planning, the writer collected some narrative texts which was done on Saturday 17th April, 2010. Borg and Gall (1983: 781) stated that a good plan can help the developer avoid much wasted work during later phases of good R & D cycle. So in this step, the writer developed a good plan before writing the test.

The writer did planning by doing operationalization, which was included selecting and specifying the materials to construct the test (Bachman’s and Palmer’s Test Development). By doing selecting and specifying the materials the writer would be able to make a blueprint of the test. A blueprint contained a test structure and test task specifications. This step would be done by conducting library study.

Research result of this step is a blueprint that can be seen in Tabel 4.2 below:

Blueprint

Test structure

a. Number of parts/tasks

The test was organized into two tasks. The tasks of this test will be cloze tasks and impromptu reading plus comprehension questions.

b. Salience of

parts Parts are clearly distinct Test task

specifications a. Purpose see the design statement (page 36-37)

(59)

Blueprint

Test task specifications

b.Definition of construct (s):

see the design statement (page 36-37)

c. Setting Pangudi Luhur St. Yosef Surakarta Senior High School

d.Time allotment 45 minutes

e. Instructions:

1) Please read the text carefully and choose the correct answer. Give cross (X) to the appropriate answer on your answer sheets.

2) Fill in blanks with the correct answer provided in number 17-20.

f. Characteristics of Input and Expected Response

There are five texts (see in the appendix 2 page 64-70) that would be used to design the test. The texts were obtained through searching on the internet because internet provided many relevant sources. Then some of the texts were simplified without changing the meaning. Four of the texts (texts 1,2,3,4, see the appendix 2 page 64) would be used to design impromptu reading plus comprehension questions and one of them (text 5, see the appendix 2 page 69) would be used to design cloze task because the writer wants to explore students comprehension based on the lesson plan. The students should answer the test by crossing the answer on the answer sheets.

g.Scoring method

In this research, the writer gave score based on the mark of correct number. If the test consists of a number of objective subsets (for example: multiple-choice), then each item may have been assigned a mark of 1 (one) if correct and 0 (zero) if wrong (Alderson, Clapham, and Wall, 1995:148).

Table 4.2: Blueprint of The Test

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(60)

3. Developing Preliminary Form of Product

Step three, developing preliminary form of product.”Developing preliminary form of product includes preparation of instructional materials handbooks, and evaluation devices.” (Borg & Gall, 1983: 775)

The research results of this step were as follows:

a. Multiple-choice items test (see the appendix 3 page 71)

The test consisted of five (5) narrative texts and twenty (20) items. Four (4) texts were used to design impromptu reading plus comprehension questions and one (1) of them would be used to design cloze task. Each item had five (5) choices named A, B, C, D, E.

b. The answer

The answer key of the test: 1. C

2. B 3. A 4. E 5. D

6. B 7. C 8. C 9. A 10.B

11.D 12.B 13.A 14.D 15.E

16.B 17.C 18.E 19.B 20.D c. Scoring and marking system

For the scoring system, the writer assigned a mark of 1 (one) if correct and 0 (zero) if wrong. The total number of correct answer would be divided by two (2) to get the final mark.

(61)

4. Preliminary Field Testing

In the fourth step, preliminary field testing, the writer conducted the test to the students of X grade of Pangudi Luhur St. Yosef Surakarta Senior High School in 2009/2010 academic year on April 29, 2010. The students were asked to read the texts and give cross (x) to the correct answer of the questions in forty-five minutes (45’).

After conducted the test, the writer obtained the answer sheets of the students, counted IF, ID, and distractor efficiency and also the result of the questionnaire. Here are the research results of Preliminary Field Testing:

a. The result of counting IF, ID, and distractor efficiency Item

No. IF ID

Number of student answer the distractor A B C D E 1. 0.40 0.56 19 103 96 16 2 2. 0.76 0.40 17 179 18 18 1 3. 0.44 0.42 105 61 30 14 25 4. 0.60 0.33 1 2 6 84 143

5. 0.03 - 0.01 3 201 23 8 1

6. 0.48 0.70 36 112 47 16 20 7. 0.57 0.59 47 8 135 15 26 8. 0.78 0.42 7 10 185 12 22 9. 0.01 0.01 3 203 12 15 3 10. 0.28 0.46 64 66 27 39 36

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

(1)

108

No. Item No

Score

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

195. B A A E B D C E B A A D A E E C B C B D 7

196. B B C D B D E C B E C B E E E B C A B E 7

197. B A D E B B D D B C C B A C E D C E C E 7

198. B C B D B C A C C D B B A E E B E A B D 7

199. A B A E B C E D B A - A B E E B B D E D 6

200. A B A E B C E C B A C A B D E C D D C C 6

201. B C B E B E - C B E C B B E E B C A E E 6

202. D A A E B D C C B D C B B E E C A E A E 6

203. B B B D B B A C B E C B A B E D B D C E 6

204. B B B D B B B C B E C B A B E A B D C E 6

205. B B C E B C - C B A D C A C E A A B D B 6

206. B B E D B E D C B C C B A E E E C C E A 6

207. B A C D B D C B B A C B A A E D C E A E 6

208. B A C D B - A B B - D B A D E D A B C D 6

209. B B - D B B A D E B E B C A E A A A C B 5

210. B B A D B A E E B A C E A C C B B E A A 5

211. B A B B B E D E B E D B B D E A B B A D 5

212. A A E E A C E E B A B B C D E A C C C E 5

213. A B B D B E A C B E C B B E E B C A E E 5

214. B C B E B D A C B E C D A E E A C C A E 5

215. B D A D B E A C A E C B A E D A B D A E 5

216. A B A D B C C B B A A B E A C B D A C C 5

217. C B B E B A A E B A B E A A E A B B A C 5

218. B B A D B C A D B C A B C E E D E B C D 5

219. B C B D B C A C B D E B A E C B C A E E 5

220. B D A D B C A D B D E C A A D A C E B E 5

221. B C D E B A C E C A C A C D E C B A A E 4

222. C D D D B C B A B C B D B C A B C A B E 4

(2)

No. Item No Score

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

223. B B B E B C E C B A C A B D A A B C C E 4

224. B B B E B C E C B A C A B D A A B C C E 4

225. B B B E B C E E B A C B B A E C C A A E 4

226. A B A E B C E C B A C A B A A A B A A E 4

227. B B B D B A A C B A E C A - B B A C C A 4

228. B C D D B A D C B A C A A A B A C B A E 3

229. B D C D B C B A B C B D B C A B C A B E 3

230. B D A D B C D A B C B D B C A B C A C E 3

231. C D C D C B E B B C B D C A B D B E C A 3

232. E A D E B A D E B C C A C E E D B A B E 3

233. A - B D B C C B B A A B A E D E B C D A 3

234. B C A D B D A D B A A C C E D A B C A D 2

235. B E B C B D A E B - - - A A C - E A A D 2

236. B B E D C A C C B D B B A D E A C B A E 2

C B A E D B C C A B D B A D E B C E B D

Correct answer

96 179 105 143 8 112 135 185 3 66 102 185 174 70 187 132 155 92 85 88

Total Students

0.40 0.75 0.44 0.60 0.03 0.47 0.57 0.78 0.01 0.27 0.43 0.78 0.73 0.29 0.79 0.55 0.65 0.38 0.36 0.37 IF Total 0.70 0.93 0.69 0.77 0.01 0.83 0.87 0.97 0.26 0.56 0.62 0.87 0.87 0.33 0.97 0.86 0.83 0.56 0.51 0.48 IF Upper 0.14 0.53 0.26 0.43 0.02 0.12 0.28 0.55 0.01 0.10 0.17 0.69 0.62 0.26 0.70 0.25 0.47 0.08 0.16 0.21 IF Lower 0.56 0.39 0.42 0,33 -0.01 0.70 0.58 0.42 0.01 0.46 0.44 0.17 0.24 0.06 0.26 0.60 0.35 0.47 0.34 0.25 ID

(3)

vii ABSTRACT

Widyasari, Ruth. 2011. The Design of an Achievement Multiple-Choice Item Narrative Reading Test for the Tenth Grade Pangudi Luhur Santo Yosef

Surakarta Senior High School. Yogyakarta: English Language Education Study

Program, Sanata Dharma University.

(4)

viii

(5)

ABSTRAK

Widyasari, Ruth. 2011. The Design of an Achievement Multiple-Choice Item Narrative Reading Test for the Tenth Grade Pangudi Luhur Santo Yosef

Surakarta Senior High School. Yogyakarta: Program Studi Pendidikan Bahasa

Inggris, Universitas Sanata Dharma.

Tes ini terdiri dari lima (5) teks bacaan narasi yang diambil dari internet dan dua puluh (20) pertanyaan dengan lima (5) pilihan. Lebih jauh lagi, penulis

(6)

The design of an achievement multiple-choice item narrative reading test for the tenth grade Pangudi Luhur Santo Yosef Surakarta Senior High School.

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

Dedication Page

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

TABLE OF CONTENTS

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

multiple-PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

#

2

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI

Parts

Dokumen yang terkait

DESIGNING ENGLISH LISTENING MATERIALS FOR THE TENTH GRADE STUDENTS OF SENIOR HIGH SCHOOL.

An analysis on a reading summative test of the tenth grade senior high schools in Yogyakarta.

The instructional reading materials using content-based instruction for the tenth grade students of Tarakanita Magelang Senior High School.

Designing a set of web-based multiple-choice reading test for the grade XII students of senior high school in Kecamatan Cibinong.

Designing an extracurricular English program for the first grade of the Senior High School.

A CONTENT ANALYSIS ON "ENGLISH ZONE" AN ENGLISH TEXTBOOK FOR THE TENTH GRADE STUDENTS OF SENIOR HIGH SCHOOL.

AN ANALYSIS OF READING MATERIALS IN THE TENTH GRADE SENIOR HIGH SCHOOL TEXTBOOK BAHASA INGGRIS YEAR X.

THE EFFECTIVENESS OF CLUSTERING TECHNIQUE FOR TEACHING WRITING NARRATIVE TEXT (An Experimental Research at the Tenth Grade of Senior High School) - repository perpustakaan

THE EFFECTIVENESS OF CLUSTERING TECHNIQUE FOR TEACHING WRITING NARRATIVE TEXT (An Experimental Research at the Tenth Grade of Senior High School) - repository perpustakaan

DESIGNING A SET OF WEB-BASED MULTIPLE-CHOICE READING TEST FOR THE GRADE XII STUDENTS OF SENIOR HIGH SCHOOL IN KECAMATAN CIBINONG A THESIS

Dokumen yang Anda mencari sudah siap untuk unduhkan