Technique of Data Analysis

F. Method of the Study

This study considered as quantitative approach due the writer use statistic method in order to analyze difficulty index and discrimination index using a statistical analysis formula which also has been stated in chapter two. This study categorized as content analysis. Because this study, studying the content of document especially in written text. To gather the document needed, the writer collects all suitable documents. Those documents are the English odd summative paper test, the softcopy of students’ answer sheet and answer key.

G. Data Description

The writer analyzes item difficulty, discriminating index, and distracter effectiveness in multiple-choice question of the English summative test. To get the data, she took the students’ answer sheets and the English odd summative test document to be analyzed. The data that the writer used in this study is the English summative test for odd semester. The total number of the items are 40 questions, with all of multiple-choice items. The test was held on Wednesday, December 11 th , 2013 with the given time 90 minutes. In doing this analysis, the writer took 27 percents of students’ answer sheet to determining upper and lower group students. For upper group students is defined as as the students who had good performed in a test, and the lower group can be defined as the students who had bad performed in a test. Therefore, from 27 percents from 161 students there are 43 students answer sheets for each upper group and lower group students. Another 75 students answeer sheets categorized as middle group students. This middle group students was put aside because it has average score. The data collected were analyzed by using some criteria in order to know whether the items of english summative test are qualified based on Arikunto’s or not. The first criterion is difficulty index, it is considered to be a good item test, if it has difficulty index range from 0.30 – 0.70. Second criterion is discrimination index, an item test considered to be a good test if it has discrimination index range from 0.4 – 0.7. The last is distracter, distracters are considered to be effective if it selected at least by 5 of examinee. That means, if the distracters are choosen less than 5, the distracters are considered to be ineffective. This principle are based on Suharsimi Arikunto’s 8 and Marrie Tarrant’s 9 principle. The write r here adopt Soenardji Djiwandono’s the distribution table of students’ responses concept 10 and use Arikunto’s formulation in order to know difficulty index, discriminating index, distacter effectiveness. Based on the analysis of students ’ answer sheet of English Odd Summative Test from upper and lower group for difficulty index, the writer found that, from 40 items, there are 6 15 acceptable items to meet the criteria of index difficulty. Besides there is 1 2.5 too difficult item and there are 33 82.5 unacceptable items due to the easiness of the items. For the discriminating index, the writer found there are 23 57.5 poor items, 1 2.5 items has negative result, which has to be discard, and 16 40 acceptable items. Moreover for the distractor effectiveness, the writer found there are 34 17 distracters with effective distractor and 166 83 distracters with in-effective distractor. See Appendix no. 5

H. Data Analysis

From the data description above, the writer make a pie chart in order to make the result more easier to read. 8 Arikunto, op. cit., p. 205. 9 Marrie Tarrant, et.a l, “An Assessment of Functioning and Non-functioning Distractors in Multiple- Choice Questions”, BMC Medical Education, 9, 2009, p. 2. 10 M. Soenardi Djiwandono. Tes Bahasa Pegangan Bagi Pengajar Bahasa, Jakarta: PT. Indeks, 2008, pp. 225-228. From the chart above, the writer assume that almost all of the test item on English summative test are easy items and poor in discriminate among the upper and lower group. This kind of result was caused by the test maker who did not know how to make a good multiple-choice item test. The writer has found out in this multiple-choice item test there are some ambiguity and unclearity both in the stem and alternatives area despite the clearity of the stems are crucial in item writing guideline. Also, not all of the alternatives fit with the stem, the test maker intentionally giving many contextual clues, and there are some structural clues. Those problems reflected in distracter effectiveness chart result. Because the test maker giving out a lot of clues like structural and contextual clues in alternatives. Therefore the lower group also choose the right answer, and this situation caused the poor discriminating index between the upper and the low. Moreover, the test maker gives too much reading requirement question in each reading section. In contrast, Kenneth said to keep the reading requirement question minimum. 11 In Evaluasi Pendidikan Evaluation for Education by Wayan and Sunartana, the pattern in placing the items is based on students psychological. It said to start the items from the easy level and continues to the hard one; the ration is 3:5:2. Three items have easy in difficulty, five items have medium difficulty, and two items have hard in difficulty 12 .

I. Data Interpretation

From the analysis result, overall this English summative test has 34 17 distracters with effective distractor. Meanwhile, the non-functioning distracters or in-effective distracters are 166 with 83 of the percentage. For the discriminating index there are 23 57.5 poor items, 1 2.5 negative result, and 16 40 acceptable items. For difficulty index, there are 33 82.5 easy items, 6 16 moderate or medium items, and 1 2.5 too difficult item. 11 Moore, op. cit., p. 270. 12 Wayan Nurkancana, Sunartana, P.P.N,. Evaluasi Pendidikan, Surabaya: Usaha Nasional, 1986, p. 55. From the percentage results, it can be said that this summative test has large amount of in-effective distracter. Because ineffective distracters makes the question easier to answer, and the result of easy question is the gap between lower and upper group become smaller. So it makes the teacher difficult to distinguished the real comprehension ability of the students. Therefore, ineffective distractors should be replaced or corrected as it affects overall the quality of the questions. Also the ratio of almost all the items in this test are easy in difficulty, it means the test did not consider the psychological of students in doing the test. Since almost all of the items are easy in difficulty and it also have many in-effective distracter, it can be said that the English summative test did not test real ability of students comprehension.