Item analysis of English summative test of Junior High School : (a case study at SMPN 11 Depok)
1
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST
OF JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A Thesis is Submitted in Partial Fulfillment of The Requirements for The Degree of
SI
Advisor:
Drs M. Farkhan, M.pd
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHING SCIENCE
SYARIF HIDAYATULLAH
STATE ISLAMIC UNIVERSITY
Jakarta
1430 H/2009 M
2
ENDORSEMENT SHEET
This “skripsi” entitled “Item Analysis of English Summative Test of
Junior High School”, written by name Evi Herawati. Students’ registration
number 102014023795 was examined. In the examination session of the faculty
of Tarbiyah and Teachers Training, Syarif Hidayatullah State Islamic
University Jakarta on 18 th of March 2009. The “skripsi” has been accepted and
declared to have fulfilled of requirements for degree of S. Pd in English
Language Education in The Department of English Education.
EXAMINATION COMMITTEE
CHAIRMAN
: Drs. Syauki M.Pd
NIP: 150 246 189
(
)
SECRETARY
: Neneng Sunengsih
NIP. 150 293 236
(
)
EXAMINER I
: A M Zaenuri, M.Pd
NIP: 150 182 900
(
)
(
)
EXAMINER II
: Dr. Atiq Susilo, M.A
NIP: 19491122 197803 1001
Acknowledged by:
Dean Tarbiya and Teachers’ Training faculty
Prof. Dr. Dede Rosyada, MA
NIP. 150.231.356
i3
ACKNOWLEDGEMENT
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST OF
JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A “Skripsi”
Presented to the Faculty of Tarbiyah and Teachers Training
In a Partial fulfillment of the requirements
For the degree of S. Pd. (Bachelor of Arts) in English Language Education
Approved by Advisor
Dr. M. Farkhan, M. Pd
NIP. 150 299 480
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHERS’ TRAINING
STATE ISLAMIC UNIVERSITY
SYARIF HIDAYATULLAH
Jakarta
i4
ACKNOWLEDGEMENT
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST OF
JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A “Skripsi”
Presented to the Faculty of Tarbiyah and Teachers Training
In a Partial fulfillment of the requirements
For the degree of S. Pd. (Bachelor of Arts) in English Language Education
Approved by Advisor
Dr. M. Farkhan, M. Pd
NIP. 150 299 480
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHERS’ TRAINING
STATE ISLAMIC UNIVERSITY
SYARIF HIDAYATULLAH
Jakarta
i5
ACKNOWLEDGEMENT
In the name of Allah, the Beneficent, the Merciful
All Praise is to Allah Lord of the worlds who has bestowed upon the writer
in completing this skripsi. Peace and blessing be upon to our prophet
Muhammad SAW, his families, his companions, and his followers.
In this occasion, the writer wants to say a lot of thanks to
her advisor
Dr. Farkhan, M. Pd. for his time, guidance, kindness, contributions, and
patience in correcting and helping her to finish this paper.
The writer also realized that she would never finish writing this paper
without the help of some people around her. Therefore, she would like to give
special gratitude to: Prof. Dr. Rosyada., M.A the dean of the faculty of
Tarbiyah and
1. Drs. Syauki M. Pd. The head of English Department.
2. All the lectures in English Department faculty of Tarbiyah and teaching
science Syarif Hidayatullah State Islamic University for sharing their
knowledge.
3. The librarian of UIN Syarif Hidayatullah, UNIKA Atma Jaya, and UNJ
for supporting the writer by allowing her to overview their book
collections.
4. Drs. Saidi., M.M. The principal of SMPN 11 Depok for giving the
writer an opportunity to carry out the research.
5. The English teacher of SMPN 11 Depok. Mrs. Bekti Sukmawati and all
students of SMPN 11, for their motivations to the writer.
6. The writer would like to express her great honor and deepest gratitude
to her beloved parents (Abdul Hamid and Rimah Istiqomah), her
beloved sisters, Ika Hamidanur and Ela Yustina who always give their
love;
support,
praying,
motivations,
contributions
and
moral
encouragement to finish her study.
7. All of her friends of 2002 English Department that cannot be mentioned
one by one, for their support, motivations all cheerful time.
ii6
Finally the writer admits that her writing is still far for being perfect,
therefore, she hopes some suggestions a critics from the reader for this simple
paper and it will have some value for her and for a better thing in their future.
The writer,
iii7
THE TABLE OF CONTENTS
ACKNOWLEDGEMENT .........................................................................
i
THE TABLE OF CONTENTS .................................................................. iii
LIST OF TABLES ..................................................................................... v
CHAPTER I
INTRODUCTION
A. Background The of Study ............................................. 1
B. Statement of The Problem ............................................ 3
C. Scope and limitation of Study ....................................... 3
D. Objective of The Study ................................................. 3
E. Method of The Study ................................................... 3
F. The Organization of Study ........................................... 4
CHAPTER II THEORETICAL FRAMEWORK
A. Test .............................................................................. 5
1. Definition of Test ................................................... 5
2. Types of Test .......................................................... 6
3. Types of Test Items ................................................ 8
B. Items analysis
1. Definition of Items Analysis ................................... 14
2. Index of Difficulty .................................................. 14
- Level of Difficulty ............................................... 14
- Discriminating Power .......................................... 15
- the Effectiveness of Distracters ............................ 15
CHAPTER III RESEARCH OF METHODOLOGY FINDINGS
A. Research Methodology ................................................ 17
1. The Purpose of Study ............................................. 17
2. Place and Time of Study ......................................... 17
3. Technique of Sample Taking .................................. 17
4. Technique of Data Collection ................................. 17
5. Technique of Data Analyzing ................................. 18
iv8
B. Research of Finding
1. Description of Data ................................................ 20
2. Data Analysis ......................................................... 21
3. Alternative of Revision ........................................... 39
CHAPTER 1V CONCLUSION AND SUGGESTION
A. Conclusion ................................................................... 43
B. Suggestion ................................................................... 43
REFERENCES .......................................................................................... 45
APPENDIX
v9
LIST OF TABLES
1. Table 1
The Students Scores of The English Summative test result.24
2. Table 2 – 9
Level of Difficulty ..…………………………..…………. 25
3. Table 10 – 17
Discriminating Power Index ..…………………………… 29
4. Table 18 – 25
the Effectiveness of Distracter ..………………………..... 34
10
CHAPTER I
INTODUCTION
A. The Background of Study
English is a tool of communication to get information and it can be used in
formal education as academic subject matter. One of the ways to know whether
students have learned their lesson at school is by evaluation. Evaluation is one
of important aspects in teaching-learning process beside of objective and
methods.
Based competence based curriculum 2004 evaluation is a set of activity
used to gain, analyze, and interpret the data about student process and learning
outcomes that are done systematically and continuously so that result will
become meaningful information for decision-making.
Evaluation is defined as a systematically process of determining the extent
to which pupils achieve instructional objectives. For the class teacher, it will be
useful to know achievement of objective and can motivate the students with
pupil achievement.1
The writer considers that the test is one of the instruments that can be used
for gathering the information about the students’ strength and the weakness in
accepting the lesson.
There are many methods for collecting information for evaluation purposes.
One of item is by using a test2. A test in plain word is “A method of measuring
a person’s ability or knowledge in a given domain”3. Test are often either used
1
Drs. M. Chabib (ed.), Teknik Evaluasi Pendidikan (Jakarta: P.T. Raya Grafindo Persada,
1996), P.1.
2
Fred Genesse and John A Upshur (eds.), Classroom Based Evaluation, (Melbourne:
Cambridge University Press, 1966), P.140.
3
H. Douglass (ed.), Teaching by principles, An Interactive Approach to Language
Pedagogy, (San Francisco: Addison Wesley Long man, 2001), P.384.
11
for pedagogical purposes, either as a means of motivating students to study or
as a means of reviewing material taught4.
Students usually tend to study harder and be motivated to study with review
material when they are going to have an examination or if the teacher
announces that they are going to have an examination. As means to measure
students’ achievement in learning process, a test should be constructed well so
that it is able to distinguish between the students who have studied well and
who have not.
Teachers are ones who know the characteristics of their classes. Thus, they
are in the best position to construct test items to measure their student
achievement and it is not an easy job. Stated by J. Stanley Ahmann and Marvin
D. Glock: “Classroom tests are test constructed by classroom teacher for use his
particular classes … more of these tests are administered than any other kind.
Unfortunately they are carelessly constructed and interpreted”5.
Based on the information above, the writer would like to see the quality of
test item by doing item analysis of English Summative Test at second year
students of SMPN 11 Depok. The total number of students is 375 students, but
the writer took only 85 students. The research is seen from the level of
difficulty, discriminating power, and the effectiveness of distracter.
A. Statement of the Problem
Based o the description above, the writer states the problem “Are the
English Summative test items tested at the second year students of SMPN 11
Depok qualified as good test items or not that will see from the level of
difficulty, discriminating power, and the effectiveness of distracters?
4
Lyle F. Bahman (ed.), Fundamental Consideration in Language Testing,(Toronto:
Oxford University Press, 1990), P.22.
5
J. Stanley Ahmann and Marvin D. Glock (eds.), Evaluating Pupil Growth, principles Of
Tests and Measurements, (Boston: Allyn and Bacon, INC, 1967), P.384.
12
C. Scope and Limitation of Study
According to Suharsimi Arikunto, “Ada empat cara yang digunakan untuk
manilai kualitas soal yaitu dengan meneliti soal yang sudah disusun untuk
mengetahui kejelasan perintah atau bahasa, dengan melakukan analisis soal,
dengan mengadakan checking validitas, dan dengan mengadakan checking
reliabilitas.” (There are four ways to see the quality of test items, they are: by
checking the test items that have been arranged to know the clarity of question
or using language, by doing item analysis, by checking validity and the last one
is by checking reliability)6. But the writer intends to see the quality of test items
only by doing item analysis that focuses on three characteristics, they are: level
of difficulty, discriminating power, and the effectiveness of distracters. The test
item that will be analyzed by the writer is objective test of English Summative
Test only that was used at the second year students of SMP 11 Depok.
D. The Objective of Study
The objective of study is to find out the level of difficulty, the
discriminating power and the effectiveness of distracter of English summative
test items tested at the second year students of SMP 11 Depok
E. The method of Study
To support the discussion, the writer takes field research by observing the
English summative test paper and the students answer sheet of the second year
of SMP 11 Depok to know the quality of the test items seen from the level of
difficulty, discriminating power and the effectiveness of distracters.
F. The Organization of Study
This “Skripsi” consists of four chapters.
The first chapter is introduction; it contains the background of study.
6
Suharsimi Arikunto (ed.), Dasar-dasar Evaluasi Pendidikan, (Jakarta: Bumi AKsara,
1997), P. 211
13
The second chapter is theoretical work, which discusses about definition of
test, types of test, types of test items, definition of items analysis, kinds of items
analysis, and the important of items analysis.
The third chapter, the writer will discuss about the purpose of study, place
and time of study, the procedure of research, the technique of data collecting
and technique of data analyze.
The fourth chapter talks about conclusion and suggestion.
14
CHAPTER II
THEORETICAL FRAMEWORK
A. TEST
1. Definition of test
According to Anthony J. Nitko:” Test is a systematic procedure for
observing and describing one or more characteristics of a person with the aid of
either a numerical scale or category system”7. While Wayan Murkancana and
PPN Sumartana stated that: “Test adalah suatu cara untuk mengadakan
penelitian yang berbentuk suatu tugas yang harus dikerjakan oleh anak atau
kelompok anak sehingga menghasilkan suatu nilai tentang tingkah laku atau
prestasi anak tersebut yang dibandingkan dengan nilai yang di capai oleh anak
lain atau dengan nilai standar yang ditetapkan”8 (Test is a technique of
measurement which can be a task or correlated tasks which must be done by a
learner or a group of learners, in order to yield a norm about his or their
achievement and can be compared with score achieved by a learner or a group
of learners, or with a standard norm).
Meanwhile Michael t. Nietzel in her book “Introduction to Clinical
Psychological” stated that “test is a systematic procedure for observing and
describing a person’s behavior in a standard situation”9
Based on the above opinion, the writer can conclude that a test is a
procedure designed to elicit score from which one can make inference about
certain characteristic of individual.
2. Types of Test
7
Anthony J. Nitko (ed.), Educational Test and Measurement An Introduction, New York
Harcourt Brace Jovanovich, Inc, 1983), P. 6
8
Wayan Murkancana and PPN Sumartana (eds.), Evaluasi Pendidikan (Surabaya:PT.
Usaha Nasional, 1982), P. 43
9
Michael T. Nietzel (ed.), Introduction to Clinical Psychology, (New Jersey: Prentice Hall,
1998), P. 159.
15
There are many types of test used to measure student’s achievement. N.E.
Gronlund said that the types of test are divided into four types of test, they are
placement, formative, diagnostic and summative test10.
a. Placement test
Placement test is a test designed to determine candidate’s place in the
teaching program at the beginning of instruction and placement evaluation is
concerned with the pupils’ entry performance and the mode of instruction that
is most likely to provide option achievement for each pupil.
b. Formative test
The formative test is a test designed to give the student immediate feedback
at the end of a unit in the course book or after a lesson designed. Formative test
is used to monitor learning progress during instruction 11. The purpose of the
formative test is to provide to continuous feedback to both pupil and teacher.
c. Summative test
Summative test is a test intended to show the standard that the students have
reached in relation to other students at the same stage and typically comes at
the end of a course or unit of instruction.
According to Norman E. Gronlund
“Summative test is designed to determine the extent to which the instructional
objective has been achieved” 12.
d. Diagnostic test
Diagnostic test is a test designed to find out learners’ strengths and
weakness with various observational techniques. Diagnostic test is much more
comprehensive and detailed because it searches for the underlying causes of
learning difficulties and then formulates plan for remedial action.
10
N. E. Gronlund, Measurement and Evaluation in Teaching 3rd ed ( New York:
Macmillan Publishing Co., 1985) p. 17
11
N. E. Gronlund, Measurement and Evaluation…, p. 17
12
N. E. Gronlund, Measurement and Evaluation…, p. 17
16
According to N. E. Gronlund “Diagnostic test is concerned with the
persistent or recurring learns of difficulties that are left unresolved by the
standard corrective prescriptions of formative evaluation” 13. While a diagnostic
test is designed to diagnose a particular aspect of a language.
e. Proficiency test
Proficiency test is a test designed to decide language proficiency of the testtakers. Arthur Hughes in his book “Testing for language teacher” said that the
content of proficiency test is not based on the content or objective of language
courses that people taking from the test may have followed. Rather, it is based
on a specification of what candidates have to be able to do in the language in
order to be considered proficiently14. Harold S. Madsen said that “proficiency
test can measure overall mastery of English or how well prepared one is to use
English in a particular setting such as an auto mechanics course or a university”
15
.
f. Achievement test
An achievement test is a test related directly to classroom lessons, units, or
even a total curriculum. Achievements tests are limited to particular material
covered in a curriculum within a particular time frame and are offered after a
course has covered.16
Achievement test is related to what they measure, what the students have
learned as a result of teaching.17 The achievement test attempts to measure the
extent to which pupil has achieved in various subject area.18
13
N. E. Gronlund, Measurement and Evaluation…, p. 17
14
Arthur Hughes, Testing for Language teacher, (Melbourne: Cambridge University press,
1991), p. 9
15
Harold S. Madsen, Technique in Testing, (New Jersey, Oxford University Press, 1983),
p. 8
16
H. Douglass, Teaching by Principles, An Introduction to Language Pedagogy, (San
Francisco: Addison Wesley Longman, 2001), p. 384
17
Tim mc Namara, language Testing, ( Hongkong: Oxford University Press, 2000), p. 7
17
There is an achievement test that is published commercially. But the
achievement test will be less interest than proficiency test because achievement
test is often prepared to measure progress only in a specific textbook series and
most teachers will prepare their own classroom progress tests.19
3. Types of Test Items
a. Subjective test
According to Arthur Hughes: “If no judgment is required on the part of the
scorer, then, the scoring is subjective … if judgment is called for, the scoring is
said to be objective”.20 It means subjective test is a test where in its scoring
requires judgment and evaluation on the field to measure the student ability and
knowledge about their receptive and productive skill.
The subjective tests are common used in classroom are essay, short answer
and completion:
1. Essay
Essays have been used among teachers for a long time and in answering the
question, the students are given freedom to select what they will explain about
the question. Essay questions can be classified into two types they are Question
and Extended Response Question 21:
•
Restricted Response Question
In the restricted response type test, the students are not given complete
freedom or restricted in making their response. For example: “State two main
18
Bill R. gearhearts and Ernest P. Wallenberg. Application of pupil assessment
Information, (Colorado: Love publishing Company, 194), p. 52
86.
19
Harold S. Madsen, Technique in …, p. 189.
20
Arthur Hughes, Testing for …, p. 19..
21
Wilmar Tinambunan, Evaluation of Students Achievement, Jakarta: Depdikbud,1988, p.
18
differences between the simple present tense and simple perfect tense”. The
answer of such question is restricted by the scope of topic to be discussed only.
•
Extended Response Question
In the extended response, students are given complete freedom in making
their responses according to their own words. The question of extended
response question is more widely. For example: Describe the strengths of essay
questions!”
Thus, the distinctive feature of essay question is freedom of response it
provides. The advantages of the essay questions:
There are advantages and disadvantages of the essay’s questions:
a. The advantages are:
1. The essay question is useful to measure the students' ability and
knowledge to organize, integrate, and express their ideas about
the subject.
2. The possibility of guessing is minimized because no options are
provided. Thus possibility of receiving score through guessing is
eliminated.
3. Constructing essay question is relatively easy
4. Constructing essay question requires less time than the other
measurement tool such as true false or multiple choice item.
b. The disadvantages are:
1. The scoring essay question is difficult
It is difficult because involves factors such as handwriting,
spelling, and grammatical mistake are influences the scoring.
2. Unreliability of scoring.
The scoring in essay question usually based on intuitive
judgment and it is difficult to maintain a common set of criteria
for all students. Sometimes some teacher gives different score
when scoring the same paper at different time and this will give
unreliable score.
19
2. Short Answer Question
The short answer item consists of a question, which can be answered with a
word or short phrase22. For example: a question that can be answered with a
word or short phrase is called …(short answer question)
Generally, teachers prefer to use the short answer type of question, probably
because they think it has some advantages. It is relatively easy to construct, it
also gives the teacher some opportunity to see how well students can express
their thought and it is not difficult to score or mark than the essay questions.23
This type of question will be more useful only in testing knowledge of facts
and quite specific information.24 However, it is difficult to phrase the short
answer question so that only one answer is correct.25
3. Completion
The completion item is a written statement that requires the examinee to
supply the correct word or short phrase in response to an incomplete sentence,
A question or a word association. Completion test can be used effectively to
measure the recall of terms, dates, and names
This type of test can be used at almost all levels. But it is difficult to phrase
an incomplete statement so that only one answer is correct, and in making the
questions, it may not too many clues are given to answer the questions. Because
if too many clues are given, the item will be too easy, and if an insufficient
number of clues are presented, the item will be ambiguous and may yield
several possibility of correct answer. .26
22
Victor H. Noll, Introduction to Educational Measurements, Bostoin: Houghton Mifflin
Company, 1965, p.138.
23
Victor H. Noll, Introduction to Educational…, p.138.
24
Victor H. Noll, Introduction to Educational…, p. 139
25
Wilmar Tinambunan, Evaluation of Students…, p. 62
Wilmar, Evaluation of Students…, p. 62
26
20
b. Objective Test
An objective test item is any item that there is only a single predictable
correct answer.27 The test is scored in such as a manner that subjective
judgment is eliminated when determining the correctness of pupil’s answer28.
Therefore, whether the item is scored by one teacher or another, today or last
week, it will yield the same score.
The objective test items commonly used in classroom test are true false,
matching, and multiple choices.
1. True false
True false item refers to alternative response item, the items asks the
students to answer with “true” if it conform
to the truth or “false” if it
essentially incorrect.29
Thus, the item provides the students with a choice of two alternatives, so
the students have a possibility to guess the answer and sometimes it will be the
right answer. Because of the random guessing to produce the correct answer,
the true false test is become less reliable than the other types of test. However,
these items are appropriate for occasional use, for example after the students
choose the two alternatives between rights and wrong, correct or incorrect, etc.
They asked to explain by writing the sentences justifying their response.30
Another advantage of constructing a true false item is that the students are
able to respond to more true false items in a given time period than any other
selection type items. 31The most common uses of true-false items are:
27
Wilmar, Evaluation of Students…, p. 62
28
Rebecca M. Falette, Modern Language Testing, New York: Harcourt Brace Javanovich
Publishers, 1997. p.8
29
Wilmar, Evaluation of Students…, p.70
30
Barbara Gross Davis., Tools for Teaching. (San Francisco: Jersey Bass Publishers,
1993), p. 243
31
Barbara Gross Davis., Tools for…, P. 243
21
1) To measure the ability to identify the correctness of statements of facts,
definition of terms and statements of principles.
2) To measure the pupil’s ability to distinguish fact from opinion.
3) To measure aspect of understanding, this is the ability to recognize cause
and affect relationship. This type of item usually contains two true
propositions in one statement, and the pupil asked to judge whether the
relationship between them is true or false.
2. Matching
The matching test item consists of two parallel columns with each word,
number or symbol in one column being match to a word, sentence or phrase on
the other column.32 This type of item is employed widely in situation where
relationships of more or less similar ideas, facts, and principles are to be
examined or judged.33 For example:
1. ( ) it is a beautiful day
a. noun
2. ( ) beauty is only Skeen deep
b. verb
3. ( ) she sing beautifully
c. adverb
4. ( ) he beautifies his house before selling it
d. adjective
This kind of test is an effective way to test students’ recognition of the
relationships between words, definitions, events, dates, categories, example,
and so on.34
Matching items are also useful in measuring students’ ability to make
association, interpretations or measure knowledge of a series of fact. Besides
that, the matching items can be used for a large quality of associated factual
32
Barbara Gross Davis., Tools for…, p. 64
33
Victor H Noll. Introduction to Educational..., p.64
34
Barbara Gross., loc.cit
22
material to be measured in small amount of space while the students’ time
needed to respond is a relatively short.35
3. Multiple Choice
A multiple choice item consists of one or more introductory sentences
followed by a list of two or more suggested responses from which the
examinees choose one as the correct answer.36 For example:
Which one of the following is kind of subjective test item?
a. short answer
b. true false
c. matching
d. multiple choice
The multiple choices items can measure both the knowledge and
understanding level. Some advantages of using multiple choice items are: the
multiple choice items are fast, easy and economical to score, and they can be
scored objectively.
The multiple choice items also have disadvantages such as: the technique of
the test only by answering the questions with choosing a, b, c, or d based on
students’ knowledge, so the students do not have or little opportunity to express
their own idea of a problem, pupils have much time to guess the answer and it
may effect on their scores, it is difficult to write successful items, and cheating
may be facilitated.37
B. ITEM ANALYSIS
1. Definition of Items Analysis
35
Wilmar, Evaluation of Students…, p. 65
36
Anthony J Nitko, Educational Test and …, p. 190
37
Kathleen M Baily, Learning About Language Assessment, (Boston: ITP An International
Thomson Publishing Company, 1988), p. 130.
23
According to Anthony J. Nitko :”Item analysis refers to the process of
collecting, summarizing and using information about individual test items,
especially information about pupils’ responses to items”38. The analysis of
student response to objective test items is a powerful tool for test improvement.
Meanwhile J. Stanley Ahmann and Marvin D. Glock said “item analysis is
reexamining each test to discover its strength and flaws”39. Item analysis can
tell us if an item was too difficult or too easy, how well it discriminated
between high and low scores on the test, and whether all the alternatives
functioned (useful) as intended or useless.
Each questions needs to function properly: otherwise, it can get a good
quality. It is like statement from nana Sudjana “Analysis butir soal atau analisis
item adalah pengkajian pertanyaan tes agar diperoleh perangkat pertanyaanpertanyaan yang lebih memiliki kualitas yang memadai”.40 (Item analysis is
analyzing the test question to get more have good quality for the questions
instrument”
2. Index of Difficulty
a. Level of Difficulty
Level of difficulty can be identified by selecting the test with percentage of
the correct answer. According to Harold S. Madsen “level of difficulty means
the percentage who answers correctly each item”.41
38
Anthony j. nitko. Educational Test and …, P. 284
39
J Stanley Ahmann and Marvin D. Glock, Evaluating Pupil …, P.184
40
Dr. Nana Sudjana, Penilaian Hasil Belajar, (Bandung: PT.Rosde Karya, 1992), P.342
41
Harold S. Madsen, Technique in Testing, (New York: Oxford University Press, 1983),
P. 184.
24
It was efficient to compare the student who performed well with larger
percentage of the correct answer and those who performed poorly with lower
percentage of the correct answer. By analyzing the students’ response to the
items, the level of difficulty of each item can be known and the information
will be helpful for teacher in identifying concepts to re-teach the study material
and giving the students feedback about their learning”42.
b. Discriminating power
“The discriminating power of a test item is an index that shows its ability to
differentiate between pupils who have achieved well (the upper group) and
those who achieved poorly (the lower group)”43. Students who performed well
on the whole test score tend to be high and students who performed badly will
get low or upper score.
c. Effectiveness of Distracter
One important aspect affecting the difficulty of multiple choice test items is
the quality of distracters. Some distracters, in fact might not be distracting at
all, and therefore serve no purpose.44
To know whether the distracters is function or nor, distracter analysis is
done that is by comparing the number of students in the upper group and the
lower group who select each incorrect alternatives. In addition, if the distracters
are not functioned, they should be rewritten or distracter.45
Harold s. Madsen stated in his book, there are three common causes of
weak distracter: fist, sometimes an item was drilled heavily in class, so almost
everyone has mastered the item, so the answer is obvious. Second, sometimes a
42
Asmawi Zaenul and Noehi Nasution, Penilaian Hasil Belaja,r, (Jakarta: UT, 2001),
p. 13.
43
Anthony J. Nitko, op. cit., p. 292.
44
Kathleen M. Bailey, Learning About Language Assessment, (Boston: ITP An
International Thomson Publishing Company, 1988), p. 130
45
Wilmar Tinambunan, Evaluation of Students …, p. 141.
25
well recognize pair is used, such as this/these, is/are, etc. even though not
everyone has controlled of these yet, and the third cause is the use of obviously
impossible distracter, for example: Did he o the work/ A. yes, he did, B. birds
eat worms, C. Train cannot fly.46
46
Harold S. Madsen, Techniques in …, p. 184
26
CHAPTER III
RESEARCH METHODOLOGY AND FINDING
A. Research Methodology
1. The Purpose of Study
The purpose of this research is to know the quality of English summative
test items at the second year students of SMPN 11 Depok seen from the level of
difficulty, discriminating power, and effectiveness of distracter.
2. Place and Time of Study
The research was held at SMPN 11 Depok that is located at Jl.
Murbai Komplek Sukatani Permai Depok. The writer did the research on
August 10th 2007, she took the test paper and the students’ answer sheet to be
analyzed.
3. Technique of Sample Taking
The writer took the sample from the second year students of SMPN 11
Depok, and the total number of the students is 375 students that are divided into
nine classes. But as a sample, she took only 85 students, they are taken from
two classes, they are class II.3 and II.5.
4. Technique of Data Collection.
To collect data based on the topic discussion, she did on observation by
visiting the school to ask for the students’ answer sheet and the paper of the
English summative test at the second year students of SMPN 11 Depok to be
analyzed.
5. Technique of data Analyzing
In this research, the writer used quantitative method to analyze the level of
difficulty, discriminating power, and the effectiveness of distracter of English
summative test items by using some statistic formulae, namely:
27
a. Level of Difficulty Analysis, by using the formula47
LD =
WL + WH
x 100%
nL + nH
Where LD
= Level of difficulty
WL
= The total number of wrong answer of the lower group
WH
= The total number of wrong answer of the high group
nL
= The number of the lower group
nH
= The number of the high group
With the classification as follows:
LD
= 25% - 75% = Good difficulty
25%
= Very easy item
75%
= Very difficult item
b. Discriminating Power Index by using the formula48:
DI =
WL + WH
n
Where DI = The discriminating Power Index
n = The number of students in the high group of lower group
With the classification as follows49:
DI =
0. 40
= Very good
0. 30 – 0. 39 = Good
0. 20 – 0. 29 = revised
0. 19
= Discarded
47
Wayan Nurkancana dan Sumartana, Evaluasi Pendidikan,(Surabaya: Usaha
NAsional,1986), P.136
48
49
Wayan Nurkancana dan Sumartana, Evaluasi…, P. 136
Mujijo, Tes Hasil Belajar, (Jakarta: Bumi Aksara, 1995), P.69
28
c. The effectiveness of distracter by using the formula50:
(H + L)
K=
D=
100%
nH + nL
25% x 1
(nH + nL)
2 x nD
Where K = Key answer
H
= The number of the high group who choose the option
L
= The number of the lower group who choose the option
nH = The number of the students in the lower group
nL = The number of students in the lower group
D
= Distracter
nD = The number of distracter
With the classification as follows:
Key answer = 25% - 75% / frequency H > frequency L = effective
< 25% / frequency H < frequency L = effective
> 75% / frequency H = frequency L = ineffective
Distracter
=
0. 6875 / frequency H < frequency L = effective
< 0. 675 / frequency H > frequency L = ineffective
B. Research Finding
1. Description of Data
The data that the writer used in this study is English summative test, which
is Ulangan Umum Semester Genap (II)51 for the second year students of SMPN
11 Depok academic 2007. The total numbers of test items are 45 test items
50
51
Wayan Murkancana, Evaluasi Pendidikan,h.143
See APPENDIX
29
which consist of 40 multiple choice test items and 5 essay test items52. The
multiple choice items consist of 21 reading items, 2 structure, 9 vocabulary
items, 2 general knowledge items, 11 language focus items. The test was held
on Monday, 14 June 2007 with the given time 90 minutes.
2. Data Analysis
No
1
2
3
4
5
6
7
8
9
10
Table 1
The Students scores of The English Summative Test Result for 95
Students of the Second Year Students of SMPN 11 Depok academic
year 2007
Score
27%
No
Score
46%
NO
SCORE
39
24
30
63
23
38
64
23
25
30
36
65
23
26
30
36
66
22
52
27
29
See APPENDIX
35
67
22
28
29
35
68
22
H
29
29
34
69
22
30
29
I
34
70
21
31
29
G
34
71
21
32
29
H
34
72
20
33
28
27%
L
O
W
E
30
The data that will be analyzed is 27% from the upper group and 27% from
the lower group and the middle group is ignored.
a. Level of Difficulty
1) Reading
31
a. Stated Detail Question
Table 2
Level of Difficulty of Stated Detail Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
13
30
65.21
Good Difficulty
1
0
1
2.17
Very Easy
3
4
0
4
8.69
Very Easy
4
20
2
22
47.82
Good Difficulty
5
3
0
3
6.52
Very Easy
7
11
0
11
23.91
Very Easy
18
6
1
7
15.21
Very Easy
20
17
13
30
65.21
Good Difficulty
21
2
1
3
6.52
Very Easy
22
3
0
3
6.52
Very Easy
24
1
0
1
2.17
Very Easy
25
1
0
1
2.17
Very Easy
26
0
0
0
0
Very Easy
31
19
7
26
56.52
Good Difficulty
32
21
4
25
54.34
Good Difficulty
Lower Group
High Group
(WL)
(HG)
1
17
2
Item
Based on the table above, we can see that in stated detail questions, there
are five items having good difficulty with level of Difficulty between 25% 27%, they are items no.1, 4, 20, 31, and 32. And the others are very easy
because the level of difficulty is under 25%, they are items no. 2, 3, 4, 5, 18, 21,
22, 24, 25, and 26.
b) Unstated Detail Questions
Table 3
Level of Difficulty of Unstated Detail Questions
32
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
23
12
8
WL+WH
LD%
Explanation
35
76.08
Very Difficulty
There is one unstated detail questions based on the data above that no. 8 is
very difficult that is 76.08%.
c) Pronoun Inferences
Table 4
Level of Difficulty of Pronoun Referents Questions
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
6
4
6
WL+WH
LD%
Explanation
10
21.73
Very Easy
There is one Pronoun Inferences based on the data above that is no. 6 with
have very easy that is 21.73%.
d) References
Table 5
Level of Difficulty of References Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
13
29
63.04
Good Difficulty
14
5
19
41.30
Good Difficulty
19
7
7
14
30.43
Good Difficulty
33
17
1
18
39.14
Good Difficulty
Lower Group
High Group
(WL)
(HG)
15
16
17
Item
There are four references based on the data above, they are no. 15, 17, 19,
and 33 and all of them have good difficulty, they are 63.04 %, 41.30%, 30.43%,
and 39.14%.
33
2). Structure
Table 6
Level of Difficulty of Structure Questions
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
3
3
38
WL+WH
LD%
Explanation
6
13.04
Very Easy
There is one structure question based on the data above no. 38 is very easy
that is 13.04%.
3). Vocabulary
Table 7
Level of Difficulty of Vocabulary Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
0
3
6.52
Very Easy
12
1
13
28.26
Good Difficulty
11
18
5
23
50
Very Easy
12
16
20
36
78.26
Very Difficulty
13
17
2
19
41.30
Very Easy
16
17
7
24
52.17
Good Difficulty
23
13
1
14
30.43
Good Difficulty
Lower Group
High Group
(WL)
(HG)
9
3
10
Item
In vocabulary there are seven items, we can see no. 12 is very difficult item
78.26%. And others are very easy level of difficulty. There are no. 9, 10, 11, 13
16, and 23.
4). General Knowledge
Table 8
Level of Difficulty of General Knowledge Questions
34
Total Wrong Answer
No.
Item
14
Lower Group
High Group
(WL)
(HG)
13
7
WL+WH
LD%
Explanation
20
43.47
Good Difficulty
Based on the data above, there is no. 14 has good difficult that is
43.47%.
5). Language Focus
Table 9
Level of Difficulty of Language Focus Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
0
9
19.56
Very Easy
16
7
23
50
Very Easy
29
9
1
10
21.73
Very Easy
30
13
3
16
34.78
Good Difficulty
34
19
4
23
50
Good Difficulty
35
12
5
17
36.95
Good Difficulty
36
17
3
20
43.47
Good Difficulty
37
16
0
16
34.78
Good Difficulty
39
15
5
21
45.65
Good Difficulty
40
21
4
25
54.34
Good Difficulty
Lower Group
High Group
(WL)
(HG)
27
9
28
Item
There are ten language focus questions based on the data above, there are
three items are very easy in level of difficulty they are no. 27, 28, 29. And
others have good difficulty they are no. 30, 34, 35, 36, 39 and 40.
b. Discriminating Power Index
1) Reading
a. Stated Detail Question
35
Table 10
Discriminating Power Index of Stated Detail Questions
No.
Total Wrong Answer
WL+WH
DP%
Explanation
13
4
0.17
Discarded
1
0
1
0.04
Discarded
3
4
0
4
0.17
Discarded
4
20
2
18
0.78
Very Good
5
3
0
3
0.13
Discarded
7
11
0
11
0.47
Very Good
18
6
1
5
0.21
Revised
20
17
13
4
0.17
Discarded
21
2
1
1
0.04
Discarded
22
3
0
3
0.13
Discarded
24
1
0
1
0.04
Discarded
25
1
0
1
0.04
Discarded
26
0
0
0
0
Discarded
31
19
7
12
0.52
Very Good
32
21
4
7
0.30
Good
Lower Group
High Group
(WL)
(HG)
1
17
2
Item
From the above data, we can see there are 15 stated detail questions. Three
of them have very good discriminating index, and they are no. 3, 7, and 31 with
discriminating power between 0.47% until 0.78%. One of them has a good
discriminating power of is 0.30% that is no. 32 and one item has to be revised
that is no. 18 with 0.21%. And 10 items must be discarded because their
discriminating power are between 0 – 0.17%, they are items no. 1, 2, 3, 5, 20,
21, 22, 24, 25, and 26.
b. Unstated Detail Question
Table 11
Discriminating Power Index of Unstated Detail Questions
36
No.
Item
8
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
23
12
WL+WH
DP%
Explanation
11
0.47
Very Good
There is one unstated detail question has very good discriminating power,
that is no. 8 with discriminating power 0.57%.
c. Pronoun Referents
Table 12
Discriminating Power Index of Pronoun Referents
No.
Item
6
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
6
4
WL+WH
DP%
Explanation
2
0.08
Discarded
We can see there is one has to be discarded in Discriminating power of
Pronoun Referents that is no. 6 with discriminating power index 0.08%.
d. Inferences
Table 13
Discriminating Power Index of Inferences
No.
Total Wrong Answer
WL+WH
DP%
Explanation
13
3
0.13
Discarded
14
5
9
0.39
Good
19
7
7
0
0
Discarded
33
17
1
16
0.69
Very Good
Lower Group
High Group
(WL)
(HG)
15
16
17
Item
Based on the data above, there are four inferences. One of them has very
good discriminating power that is no. 33 with discriminating power 0.69%.
Moreover, one has good of discriminating power that is no. 17 with 0.39%. The
others two item that have to be discarded that is no. 15 and 19 with
discriminating power 0.13 and 0.
37
2) Structure
Table 14
Discriminating Power Index of Structure
No.
Item
38
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
3
3
WL+WH
DP%
Explanation
0
0
Discarded
From the above data, there is one structure in discriminating power index
that has to be discarded, that is no. 38 with discriminating power index 0%.
3) Vocabulary
Table 15
Discriminating Power Index of Vocabulary
No.
Total Wrong Answer
WL+WH
DP%
Explanation
0
3
0.13
Discarded
12
1
11
0.47
Very Good
11
18
5
13
0.56
Very Good
12
16
20
-4
-0.17
Discarded
13
17
2
15
0.65
Very Good
16
17
7
10
0.43
Very Good
23
13
1
12
0.52
Very Good
Lower Group
High Group
(WL)
(HG)
9
3
10
Item
Based on the data, we can see there are seven vocabularies. Two of them
have to be discarded that are no. 9 with 0.13% and no. 12 with -0.17%. But the
others have very good discriminating power, they are no. 10, 11, 13, 16 and 23
with discriminating power index from 0.43% – 0.65%.
4) General Language
Table 16
38
Discriminating Power Index of General Language
No.
Item
14
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
13
7
WL+WH
DP%
Explanation
9
0.39
Good
Based on the data above, there is one has good discriminating power index
of pronoun references that is no. 14 with 0.39%.
5) Language Focus
Table 17
Discriminating Power Index of Language Focus
No.
Total Wrong Answer
WL+WH
DP%
Explanation
0
9
0.39
Good
16
7
9
0.39
Good
29
9
1
8
0.34
Good
30
13
3
10
0.43
Very Good
34
19
4
15
0.65
Very Good
35
12
5
7
0.30
Good
36
17
3
14
0.60
Very Good
37
16
0
16
0.69
Very Good
39
15
5
9
0.39
Good
40
21
4
7
0.30
Good
Lower Group
High Group
(WL)
(HG)
27
9
28
Item
Based the data above, there are 10 language focuses. There are four items
having very good discriminating power index, the items are no. 30, 34, 36, and
37, with discriminating power index 0.43% - 0.69%. And the others are good;
they are no. 27, 28, 29, 35, 39, and 40, with discriminating power index 0.30%
- 0.39%.
39
c. The Effectiveness of Distracter
1). Reading
a. Stated Detail Question
Table 18
The Effectiveness of Distracter of Stated Detail Questions
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
1
A
3
7
Effective
B
9
8
Ineffective
C*
10
7
Effective
D
1
1
Ineffective
2
A
B*
Ineffective
23
22
C
3
5
Ineffective
D
1
Effective
A
1
Effective
B
1
Effective
19
Effective
D
2
Effective
A
1
Effective
C*
4
23
B
3
17
Effective
C
7
2
Ineffective
D*
13
3
Effective
A*
23
20
Effective
1
Effective
B
C
Ineffective
D
7
Effective
A*
B
23
1
Effective
12
Effective
1
Effective
40
18
C
7
Effective
D
3
Effective
A
1
5
Effective
B*
22
17
Effective
1
Effective
C
D
20
21
Ineffective
A
9
14
Effective
B
2
3
Effective
C
2
D*
10
Effective
6
Effective
A
1
Effective
B
1
Effective
21
Effective
C*
23
D
22
A*
Ineffective
23
20
B
Ineffective
C
24
3
Ineffective
A
Ineffective
23
22
C
1
A
Effective
Ineffective
B
C*
Effective
Ineffective
D
26
Effective
D
B*
25
Effective
23
1
Effective
22
Effective
D
Ineffective
A
Ineffective
B*
23
23
Ineffective
C
Ineffective
D
Ineffective
41
31
A
5
B
32
11
Effective
6
Effective
C*
16
4
Effective
D
2
2
Ineffective
A
1
9
Effective
B
1
3
Effective
C*
19
2
Effective
D
2
9
Effective
Based on the above, we can see there are 60 answers key from 15 the
answer key of stated detail question items they are 26 answer keys of the
effectiveness of distracters and 20 answer keys ineffectiveness of distracters.
b. Unstated Detail Question
Table 19
The Effectiveness of Distracter of Unstated Detail Questions
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
8
A
2
6
Effective
B
2
8
Effective
C
8
9
Effective
D*
11
Effective
The table above shows all of the answer keys are effectiveness of
distracters.
c. Pronoun Referents
Table 20
The Effectiveness of Distracter of Pronoun Referents Questions
No. Item
Option
6
A
High Group (H)
Lower Group (L)
Explanation
Ineffective
B*
19
17
Effective
C
1
5
Effective
42
D
3
1
Effective
The table show there is one answer key in ineffectiveness of distracter and
two answer keys are effectiveness of distracters.
d. Inferences
Table 21
The Effectiveness of Distracter of Inferences Questions
No. Item
Option
High Group (H)
15
A
9
B
1
7
Effective
C
7
9
Effective
D*
6
7
Ineffective
A
2
8
Effective
B*
18
9
Effective
C
3
1
Ineffective
5
Effective
17
Lower Group (L)
Ineffective
D
19
33
Explanation
A
1
Ineffective
B
1
1
Ineffective
C*
16
16
Ineffective
D
5
6
Effective
8
Effective
A
B*
15
6
Effective
C
6
6
Ineffective
D
2
3
Effective
Based on the data above, we can see that there are 16 answer keys from 4
stated detail question items they are 7 the answer keys of effectiveness of
distracter and 5 answer keys of ineffectiveness of distracters.
2) Structure
Table 22
The Effectiveness of Distracter of Structure
43
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
38
A*
20
20
Ineffective
B
Ineffective
C
1
D
2
Effective
3
Effective
From the data above, we can see that there are two the answer keys of
effectiveness of distracters and one in ineffectiveness of distracter.
3) Vocabulary
Table 23
The Effectiveness of Distracter of Vocabulary
No. Item
Option
9
A
10
11
12
High Group (H)
Explanation
2
Effective
B
Ineffective
C
Ineffective
D*
23
21
Effective
A
22
5
Ineffective
B*
1
11
Effective
C
5
Effective
D
2
Effective
A
2
6
Effective
B
1
10
Effective
C
2
2
Ineffective
D*
18
5
Effective
A
1
7
Effective
B
19
6
Ineffective
2
Effective
7
Ineffective
14
Effective
2
Ineffective
C
D*
13
Lower Group (L)
3
A
B
2
44
C*
21
D
16
A*
16
B
23
6
Effective
1
Effective
6
Effective
8
Effective
C
3
7
Effective
D
4
2
Ineffective
11
Effective
A
B
Ineffective
C*
22
9
Effective
D
1
3
Effective
Based on the data above, there are 28 answer keys from 7 vocabulary items,
they are 13 answer keys of effectiveness of distracters and 8 answer keys of
ineffectiveness of distracters.
4) General Knowledge
Table 24
The Effectiveness of Distracter of General Knowledge
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
14
A
7
10
Effective
B
1
Effective
C
2
Effective
10
Effective
D*
16
We can see that all of the answer keys are effectiveness of distracters.
5) Language Focus
Table 25
The Effectiveness of Distracter of Language Focus
No. Item
Option
27
Lower Group (L)
Explanation
A
3
Effective
B
3
Effective
14
Effective
C*
High Group (H)
23
45
D
28
2
5
Effective
B*
16
7
Effective
4
Effective
D
5
7
Effective
A*
22
14
Effective
1
Effective
7
Effective
1
Effective
11
Effective
B
1
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST
OF JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A Thesis is Submitted in Partial Fulfillment of The Requirements for The Degree of
SI
Advisor:
Drs M. Farkhan, M.pd
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHING SCIENCE
SYARIF HIDAYATULLAH
STATE ISLAMIC UNIVERSITY
Jakarta
1430 H/2009 M
2
ENDORSEMENT SHEET
This “skripsi” entitled “Item Analysis of English Summative Test of
Junior High School”, written by name Evi Herawati. Students’ registration
number 102014023795 was examined. In the examination session of the faculty
of Tarbiyah and Teachers Training, Syarif Hidayatullah State Islamic
University Jakarta on 18 th of March 2009. The “skripsi” has been accepted and
declared to have fulfilled of requirements for degree of S. Pd in English
Language Education in The Department of English Education.
EXAMINATION COMMITTEE
CHAIRMAN
: Drs. Syauki M.Pd
NIP: 150 246 189
(
)
SECRETARY
: Neneng Sunengsih
NIP. 150 293 236
(
)
EXAMINER I
: A M Zaenuri, M.Pd
NIP: 150 182 900
(
)
(
)
EXAMINER II
: Dr. Atiq Susilo, M.A
NIP: 19491122 197803 1001
Acknowledged by:
Dean Tarbiya and Teachers’ Training faculty
Prof. Dr. Dede Rosyada, MA
NIP. 150.231.356
i3
ACKNOWLEDGEMENT
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST OF
JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A “Skripsi”
Presented to the Faculty of Tarbiyah and Teachers Training
In a Partial fulfillment of the requirements
For the degree of S. Pd. (Bachelor of Arts) in English Language Education
Approved by Advisor
Dr. M. Farkhan, M. Pd
NIP. 150 299 480
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHERS’ TRAINING
STATE ISLAMIC UNIVERSITY
SYARIF HIDAYATULLAH
Jakarta
i4
ACKNOWLEDGEMENT
ITEM ANALYSIS OF ENGLISH SUMMATIVE TEST OF
JUNIOR HIGH SCHOOL
(A Case Study at SMPN 11 Depok)
A “Skripsi”
Presented to the Faculty of Tarbiyah and Teachers Training
In a Partial fulfillment of the requirements
For the degree of S. Pd. (Bachelor of Arts) in English Language Education
Approved by Advisor
Dr. M. Farkhan, M. Pd
NIP. 150 299 480
By:
Evi Herawati
NIM.10201402379
ENGLISH DEPARTMENT
FACULTY OF TARBIYA AND TEACHERS’ TRAINING
STATE ISLAMIC UNIVERSITY
SYARIF HIDAYATULLAH
Jakarta
i5
ACKNOWLEDGEMENT
In the name of Allah, the Beneficent, the Merciful
All Praise is to Allah Lord of the worlds who has bestowed upon the writer
in completing this skripsi. Peace and blessing be upon to our prophet
Muhammad SAW, his families, his companions, and his followers.
In this occasion, the writer wants to say a lot of thanks to
her advisor
Dr. Farkhan, M. Pd. for his time, guidance, kindness, contributions, and
patience in correcting and helping her to finish this paper.
The writer also realized that she would never finish writing this paper
without the help of some people around her. Therefore, she would like to give
special gratitude to: Prof. Dr. Rosyada., M.A the dean of the faculty of
Tarbiyah and
1. Drs. Syauki M. Pd. The head of English Department.
2. All the lectures in English Department faculty of Tarbiyah and teaching
science Syarif Hidayatullah State Islamic University for sharing their
knowledge.
3. The librarian of UIN Syarif Hidayatullah, UNIKA Atma Jaya, and UNJ
for supporting the writer by allowing her to overview their book
collections.
4. Drs. Saidi., M.M. The principal of SMPN 11 Depok for giving the
writer an opportunity to carry out the research.
5. The English teacher of SMPN 11 Depok. Mrs. Bekti Sukmawati and all
students of SMPN 11, for their motivations to the writer.
6. The writer would like to express her great honor and deepest gratitude
to her beloved parents (Abdul Hamid and Rimah Istiqomah), her
beloved sisters, Ika Hamidanur and Ela Yustina who always give their
love;
support,
praying,
motivations,
contributions
and
moral
encouragement to finish her study.
7. All of her friends of 2002 English Department that cannot be mentioned
one by one, for their support, motivations all cheerful time.
ii6
Finally the writer admits that her writing is still far for being perfect,
therefore, she hopes some suggestions a critics from the reader for this simple
paper and it will have some value for her and for a better thing in their future.
The writer,
iii7
THE TABLE OF CONTENTS
ACKNOWLEDGEMENT .........................................................................
i
THE TABLE OF CONTENTS .................................................................. iii
LIST OF TABLES ..................................................................................... v
CHAPTER I
INTRODUCTION
A. Background The of Study ............................................. 1
B. Statement of The Problem ............................................ 3
C. Scope and limitation of Study ....................................... 3
D. Objective of The Study ................................................. 3
E. Method of The Study ................................................... 3
F. The Organization of Study ........................................... 4
CHAPTER II THEORETICAL FRAMEWORK
A. Test .............................................................................. 5
1. Definition of Test ................................................... 5
2. Types of Test .......................................................... 6
3. Types of Test Items ................................................ 8
B. Items analysis
1. Definition of Items Analysis ................................... 14
2. Index of Difficulty .................................................. 14
- Level of Difficulty ............................................... 14
- Discriminating Power .......................................... 15
- the Effectiveness of Distracters ............................ 15
CHAPTER III RESEARCH OF METHODOLOGY FINDINGS
A. Research Methodology ................................................ 17
1. The Purpose of Study ............................................. 17
2. Place and Time of Study ......................................... 17
3. Technique of Sample Taking .................................. 17
4. Technique of Data Collection ................................. 17
5. Technique of Data Analyzing ................................. 18
iv8
B. Research of Finding
1. Description of Data ................................................ 20
2. Data Analysis ......................................................... 21
3. Alternative of Revision ........................................... 39
CHAPTER 1V CONCLUSION AND SUGGESTION
A. Conclusion ................................................................... 43
B. Suggestion ................................................................... 43
REFERENCES .......................................................................................... 45
APPENDIX
v9
LIST OF TABLES
1. Table 1
The Students Scores of The English Summative test result.24
2. Table 2 – 9
Level of Difficulty ..…………………………..…………. 25
3. Table 10 – 17
Discriminating Power Index ..…………………………… 29
4. Table 18 – 25
the Effectiveness of Distracter ..………………………..... 34
10
CHAPTER I
INTODUCTION
A. The Background of Study
English is a tool of communication to get information and it can be used in
formal education as academic subject matter. One of the ways to know whether
students have learned their lesson at school is by evaluation. Evaluation is one
of important aspects in teaching-learning process beside of objective and
methods.
Based competence based curriculum 2004 evaluation is a set of activity
used to gain, analyze, and interpret the data about student process and learning
outcomes that are done systematically and continuously so that result will
become meaningful information for decision-making.
Evaluation is defined as a systematically process of determining the extent
to which pupils achieve instructional objectives. For the class teacher, it will be
useful to know achievement of objective and can motivate the students with
pupil achievement.1
The writer considers that the test is one of the instruments that can be used
for gathering the information about the students’ strength and the weakness in
accepting the lesson.
There are many methods for collecting information for evaluation purposes.
One of item is by using a test2. A test in plain word is “A method of measuring
a person’s ability or knowledge in a given domain”3. Test are often either used
1
Drs. M. Chabib (ed.), Teknik Evaluasi Pendidikan (Jakarta: P.T. Raya Grafindo Persada,
1996), P.1.
2
Fred Genesse and John A Upshur (eds.), Classroom Based Evaluation, (Melbourne:
Cambridge University Press, 1966), P.140.
3
H. Douglass (ed.), Teaching by principles, An Interactive Approach to Language
Pedagogy, (San Francisco: Addison Wesley Long man, 2001), P.384.
11
for pedagogical purposes, either as a means of motivating students to study or
as a means of reviewing material taught4.
Students usually tend to study harder and be motivated to study with review
material when they are going to have an examination or if the teacher
announces that they are going to have an examination. As means to measure
students’ achievement in learning process, a test should be constructed well so
that it is able to distinguish between the students who have studied well and
who have not.
Teachers are ones who know the characteristics of their classes. Thus, they
are in the best position to construct test items to measure their student
achievement and it is not an easy job. Stated by J. Stanley Ahmann and Marvin
D. Glock: “Classroom tests are test constructed by classroom teacher for use his
particular classes … more of these tests are administered than any other kind.
Unfortunately they are carelessly constructed and interpreted”5.
Based on the information above, the writer would like to see the quality of
test item by doing item analysis of English Summative Test at second year
students of SMPN 11 Depok. The total number of students is 375 students, but
the writer took only 85 students. The research is seen from the level of
difficulty, discriminating power, and the effectiveness of distracter.
A. Statement of the Problem
Based o the description above, the writer states the problem “Are the
English Summative test items tested at the second year students of SMPN 11
Depok qualified as good test items or not that will see from the level of
difficulty, discriminating power, and the effectiveness of distracters?
4
Lyle F. Bahman (ed.), Fundamental Consideration in Language Testing,(Toronto:
Oxford University Press, 1990), P.22.
5
J. Stanley Ahmann and Marvin D. Glock (eds.), Evaluating Pupil Growth, principles Of
Tests and Measurements, (Boston: Allyn and Bacon, INC, 1967), P.384.
12
C. Scope and Limitation of Study
According to Suharsimi Arikunto, “Ada empat cara yang digunakan untuk
manilai kualitas soal yaitu dengan meneliti soal yang sudah disusun untuk
mengetahui kejelasan perintah atau bahasa, dengan melakukan analisis soal,
dengan mengadakan checking validitas, dan dengan mengadakan checking
reliabilitas.” (There are four ways to see the quality of test items, they are: by
checking the test items that have been arranged to know the clarity of question
or using language, by doing item analysis, by checking validity and the last one
is by checking reliability)6. But the writer intends to see the quality of test items
only by doing item analysis that focuses on three characteristics, they are: level
of difficulty, discriminating power, and the effectiveness of distracters. The test
item that will be analyzed by the writer is objective test of English Summative
Test only that was used at the second year students of SMP 11 Depok.
D. The Objective of Study
The objective of study is to find out the level of difficulty, the
discriminating power and the effectiveness of distracter of English summative
test items tested at the second year students of SMP 11 Depok
E. The method of Study
To support the discussion, the writer takes field research by observing the
English summative test paper and the students answer sheet of the second year
of SMP 11 Depok to know the quality of the test items seen from the level of
difficulty, discriminating power and the effectiveness of distracters.
F. The Organization of Study
This “Skripsi” consists of four chapters.
The first chapter is introduction; it contains the background of study.
6
Suharsimi Arikunto (ed.), Dasar-dasar Evaluasi Pendidikan, (Jakarta: Bumi AKsara,
1997), P. 211
13
The second chapter is theoretical work, which discusses about definition of
test, types of test, types of test items, definition of items analysis, kinds of items
analysis, and the important of items analysis.
The third chapter, the writer will discuss about the purpose of study, place
and time of study, the procedure of research, the technique of data collecting
and technique of data analyze.
The fourth chapter talks about conclusion and suggestion.
14
CHAPTER II
THEORETICAL FRAMEWORK
A. TEST
1. Definition of test
According to Anthony J. Nitko:” Test is a systematic procedure for
observing and describing one or more characteristics of a person with the aid of
either a numerical scale or category system”7. While Wayan Murkancana and
PPN Sumartana stated that: “Test adalah suatu cara untuk mengadakan
penelitian yang berbentuk suatu tugas yang harus dikerjakan oleh anak atau
kelompok anak sehingga menghasilkan suatu nilai tentang tingkah laku atau
prestasi anak tersebut yang dibandingkan dengan nilai yang di capai oleh anak
lain atau dengan nilai standar yang ditetapkan”8 (Test is a technique of
measurement which can be a task or correlated tasks which must be done by a
learner or a group of learners, in order to yield a norm about his or their
achievement and can be compared with score achieved by a learner or a group
of learners, or with a standard norm).
Meanwhile Michael t. Nietzel in her book “Introduction to Clinical
Psychological” stated that “test is a systematic procedure for observing and
describing a person’s behavior in a standard situation”9
Based on the above opinion, the writer can conclude that a test is a
procedure designed to elicit score from which one can make inference about
certain characteristic of individual.
2. Types of Test
7
Anthony J. Nitko (ed.), Educational Test and Measurement An Introduction, New York
Harcourt Brace Jovanovich, Inc, 1983), P. 6
8
Wayan Murkancana and PPN Sumartana (eds.), Evaluasi Pendidikan (Surabaya:PT.
Usaha Nasional, 1982), P. 43
9
Michael T. Nietzel (ed.), Introduction to Clinical Psychology, (New Jersey: Prentice Hall,
1998), P. 159.
15
There are many types of test used to measure student’s achievement. N.E.
Gronlund said that the types of test are divided into four types of test, they are
placement, formative, diagnostic and summative test10.
a. Placement test
Placement test is a test designed to determine candidate’s place in the
teaching program at the beginning of instruction and placement evaluation is
concerned with the pupils’ entry performance and the mode of instruction that
is most likely to provide option achievement for each pupil.
b. Formative test
The formative test is a test designed to give the student immediate feedback
at the end of a unit in the course book or after a lesson designed. Formative test
is used to monitor learning progress during instruction 11. The purpose of the
formative test is to provide to continuous feedback to both pupil and teacher.
c. Summative test
Summative test is a test intended to show the standard that the students have
reached in relation to other students at the same stage and typically comes at
the end of a course or unit of instruction.
According to Norman E. Gronlund
“Summative test is designed to determine the extent to which the instructional
objective has been achieved” 12.
d. Diagnostic test
Diagnostic test is a test designed to find out learners’ strengths and
weakness with various observational techniques. Diagnostic test is much more
comprehensive and detailed because it searches for the underlying causes of
learning difficulties and then formulates plan for remedial action.
10
N. E. Gronlund, Measurement and Evaluation in Teaching 3rd ed ( New York:
Macmillan Publishing Co., 1985) p. 17
11
N. E. Gronlund, Measurement and Evaluation…, p. 17
12
N. E. Gronlund, Measurement and Evaluation…, p. 17
16
According to N. E. Gronlund “Diagnostic test is concerned with the
persistent or recurring learns of difficulties that are left unresolved by the
standard corrective prescriptions of formative evaluation” 13. While a diagnostic
test is designed to diagnose a particular aspect of a language.
e. Proficiency test
Proficiency test is a test designed to decide language proficiency of the testtakers. Arthur Hughes in his book “Testing for language teacher” said that the
content of proficiency test is not based on the content or objective of language
courses that people taking from the test may have followed. Rather, it is based
on a specification of what candidates have to be able to do in the language in
order to be considered proficiently14. Harold S. Madsen said that “proficiency
test can measure overall mastery of English or how well prepared one is to use
English in a particular setting such as an auto mechanics course or a university”
15
.
f. Achievement test
An achievement test is a test related directly to classroom lessons, units, or
even a total curriculum. Achievements tests are limited to particular material
covered in a curriculum within a particular time frame and are offered after a
course has covered.16
Achievement test is related to what they measure, what the students have
learned as a result of teaching.17 The achievement test attempts to measure the
extent to which pupil has achieved in various subject area.18
13
N. E. Gronlund, Measurement and Evaluation…, p. 17
14
Arthur Hughes, Testing for Language teacher, (Melbourne: Cambridge University press,
1991), p. 9
15
Harold S. Madsen, Technique in Testing, (New Jersey, Oxford University Press, 1983),
p. 8
16
H. Douglass, Teaching by Principles, An Introduction to Language Pedagogy, (San
Francisco: Addison Wesley Longman, 2001), p. 384
17
Tim mc Namara, language Testing, ( Hongkong: Oxford University Press, 2000), p. 7
17
There is an achievement test that is published commercially. But the
achievement test will be less interest than proficiency test because achievement
test is often prepared to measure progress only in a specific textbook series and
most teachers will prepare their own classroom progress tests.19
3. Types of Test Items
a. Subjective test
According to Arthur Hughes: “If no judgment is required on the part of the
scorer, then, the scoring is subjective … if judgment is called for, the scoring is
said to be objective”.20 It means subjective test is a test where in its scoring
requires judgment and evaluation on the field to measure the student ability and
knowledge about their receptive and productive skill.
The subjective tests are common used in classroom are essay, short answer
and completion:
1. Essay
Essays have been used among teachers for a long time and in answering the
question, the students are given freedom to select what they will explain about
the question. Essay questions can be classified into two types they are Question
and Extended Response Question 21:
•
Restricted Response Question
In the restricted response type test, the students are not given complete
freedom or restricted in making their response. For example: “State two main
18
Bill R. gearhearts and Ernest P. Wallenberg. Application of pupil assessment
Information, (Colorado: Love publishing Company, 194), p. 52
86.
19
Harold S. Madsen, Technique in …, p. 189.
20
Arthur Hughes, Testing for …, p. 19..
21
Wilmar Tinambunan, Evaluation of Students Achievement, Jakarta: Depdikbud,1988, p.
18
differences between the simple present tense and simple perfect tense”. The
answer of such question is restricted by the scope of topic to be discussed only.
•
Extended Response Question
In the extended response, students are given complete freedom in making
their responses according to their own words. The question of extended
response question is more widely. For example: Describe the strengths of essay
questions!”
Thus, the distinctive feature of essay question is freedom of response it
provides. The advantages of the essay questions:
There are advantages and disadvantages of the essay’s questions:
a. The advantages are:
1. The essay question is useful to measure the students' ability and
knowledge to organize, integrate, and express their ideas about
the subject.
2. The possibility of guessing is minimized because no options are
provided. Thus possibility of receiving score through guessing is
eliminated.
3. Constructing essay question is relatively easy
4. Constructing essay question requires less time than the other
measurement tool such as true false or multiple choice item.
b. The disadvantages are:
1. The scoring essay question is difficult
It is difficult because involves factors such as handwriting,
spelling, and grammatical mistake are influences the scoring.
2. Unreliability of scoring.
The scoring in essay question usually based on intuitive
judgment and it is difficult to maintain a common set of criteria
for all students. Sometimes some teacher gives different score
when scoring the same paper at different time and this will give
unreliable score.
19
2. Short Answer Question
The short answer item consists of a question, which can be answered with a
word or short phrase22. For example: a question that can be answered with a
word or short phrase is called …(short answer question)
Generally, teachers prefer to use the short answer type of question, probably
because they think it has some advantages. It is relatively easy to construct, it
also gives the teacher some opportunity to see how well students can express
their thought and it is not difficult to score or mark than the essay questions.23
This type of question will be more useful only in testing knowledge of facts
and quite specific information.24 However, it is difficult to phrase the short
answer question so that only one answer is correct.25
3. Completion
The completion item is a written statement that requires the examinee to
supply the correct word or short phrase in response to an incomplete sentence,
A question or a word association. Completion test can be used effectively to
measure the recall of terms, dates, and names
This type of test can be used at almost all levels. But it is difficult to phrase
an incomplete statement so that only one answer is correct, and in making the
questions, it may not too many clues are given to answer the questions. Because
if too many clues are given, the item will be too easy, and if an insufficient
number of clues are presented, the item will be ambiguous and may yield
several possibility of correct answer. .26
22
Victor H. Noll, Introduction to Educational Measurements, Bostoin: Houghton Mifflin
Company, 1965, p.138.
23
Victor H. Noll, Introduction to Educational…, p.138.
24
Victor H. Noll, Introduction to Educational…, p. 139
25
Wilmar Tinambunan, Evaluation of Students…, p. 62
Wilmar, Evaluation of Students…, p. 62
26
20
b. Objective Test
An objective test item is any item that there is only a single predictable
correct answer.27 The test is scored in such as a manner that subjective
judgment is eliminated when determining the correctness of pupil’s answer28.
Therefore, whether the item is scored by one teacher or another, today or last
week, it will yield the same score.
The objective test items commonly used in classroom test are true false,
matching, and multiple choices.
1. True false
True false item refers to alternative response item, the items asks the
students to answer with “true” if it conform
to the truth or “false” if it
essentially incorrect.29
Thus, the item provides the students with a choice of two alternatives, so
the students have a possibility to guess the answer and sometimes it will be the
right answer. Because of the random guessing to produce the correct answer,
the true false test is become less reliable than the other types of test. However,
these items are appropriate for occasional use, for example after the students
choose the two alternatives between rights and wrong, correct or incorrect, etc.
They asked to explain by writing the sentences justifying their response.30
Another advantage of constructing a true false item is that the students are
able to respond to more true false items in a given time period than any other
selection type items. 31The most common uses of true-false items are:
27
Wilmar, Evaluation of Students…, p. 62
28
Rebecca M. Falette, Modern Language Testing, New York: Harcourt Brace Javanovich
Publishers, 1997. p.8
29
Wilmar, Evaluation of Students…, p.70
30
Barbara Gross Davis., Tools for Teaching. (San Francisco: Jersey Bass Publishers,
1993), p. 243
31
Barbara Gross Davis., Tools for…, P. 243
21
1) To measure the ability to identify the correctness of statements of facts,
definition of terms and statements of principles.
2) To measure the pupil’s ability to distinguish fact from opinion.
3) To measure aspect of understanding, this is the ability to recognize cause
and affect relationship. This type of item usually contains two true
propositions in one statement, and the pupil asked to judge whether the
relationship between them is true or false.
2. Matching
The matching test item consists of two parallel columns with each word,
number or symbol in one column being match to a word, sentence or phrase on
the other column.32 This type of item is employed widely in situation where
relationships of more or less similar ideas, facts, and principles are to be
examined or judged.33 For example:
1. ( ) it is a beautiful day
a. noun
2. ( ) beauty is only Skeen deep
b. verb
3. ( ) she sing beautifully
c. adverb
4. ( ) he beautifies his house before selling it
d. adjective
This kind of test is an effective way to test students’ recognition of the
relationships between words, definitions, events, dates, categories, example,
and so on.34
Matching items are also useful in measuring students’ ability to make
association, interpretations or measure knowledge of a series of fact. Besides
that, the matching items can be used for a large quality of associated factual
32
Barbara Gross Davis., Tools for…, p. 64
33
Victor H Noll. Introduction to Educational..., p.64
34
Barbara Gross., loc.cit
22
material to be measured in small amount of space while the students’ time
needed to respond is a relatively short.35
3. Multiple Choice
A multiple choice item consists of one or more introductory sentences
followed by a list of two or more suggested responses from which the
examinees choose one as the correct answer.36 For example:
Which one of the following is kind of subjective test item?
a. short answer
b. true false
c. matching
d. multiple choice
The multiple choices items can measure both the knowledge and
understanding level. Some advantages of using multiple choice items are: the
multiple choice items are fast, easy and economical to score, and they can be
scored objectively.
The multiple choice items also have disadvantages such as: the technique of
the test only by answering the questions with choosing a, b, c, or d based on
students’ knowledge, so the students do not have or little opportunity to express
their own idea of a problem, pupils have much time to guess the answer and it
may effect on their scores, it is difficult to write successful items, and cheating
may be facilitated.37
B. ITEM ANALYSIS
1. Definition of Items Analysis
35
Wilmar, Evaluation of Students…, p. 65
36
Anthony J Nitko, Educational Test and …, p. 190
37
Kathleen M Baily, Learning About Language Assessment, (Boston: ITP An International
Thomson Publishing Company, 1988), p. 130.
23
According to Anthony J. Nitko :”Item analysis refers to the process of
collecting, summarizing and using information about individual test items,
especially information about pupils’ responses to items”38. The analysis of
student response to objective test items is a powerful tool for test improvement.
Meanwhile J. Stanley Ahmann and Marvin D. Glock said “item analysis is
reexamining each test to discover its strength and flaws”39. Item analysis can
tell us if an item was too difficult or too easy, how well it discriminated
between high and low scores on the test, and whether all the alternatives
functioned (useful) as intended or useless.
Each questions needs to function properly: otherwise, it can get a good
quality. It is like statement from nana Sudjana “Analysis butir soal atau analisis
item adalah pengkajian pertanyaan tes agar diperoleh perangkat pertanyaanpertanyaan yang lebih memiliki kualitas yang memadai”.40 (Item analysis is
analyzing the test question to get more have good quality for the questions
instrument”
2. Index of Difficulty
a. Level of Difficulty
Level of difficulty can be identified by selecting the test with percentage of
the correct answer. According to Harold S. Madsen “level of difficulty means
the percentage who answers correctly each item”.41
38
Anthony j. nitko. Educational Test and …, P. 284
39
J Stanley Ahmann and Marvin D. Glock, Evaluating Pupil …, P.184
40
Dr. Nana Sudjana, Penilaian Hasil Belajar, (Bandung: PT.Rosde Karya, 1992), P.342
41
Harold S. Madsen, Technique in Testing, (New York: Oxford University Press, 1983),
P. 184.
24
It was efficient to compare the student who performed well with larger
percentage of the correct answer and those who performed poorly with lower
percentage of the correct answer. By analyzing the students’ response to the
items, the level of difficulty of each item can be known and the information
will be helpful for teacher in identifying concepts to re-teach the study material
and giving the students feedback about their learning”42.
b. Discriminating power
“The discriminating power of a test item is an index that shows its ability to
differentiate between pupils who have achieved well (the upper group) and
those who achieved poorly (the lower group)”43. Students who performed well
on the whole test score tend to be high and students who performed badly will
get low or upper score.
c. Effectiveness of Distracter
One important aspect affecting the difficulty of multiple choice test items is
the quality of distracters. Some distracters, in fact might not be distracting at
all, and therefore serve no purpose.44
To know whether the distracters is function or nor, distracter analysis is
done that is by comparing the number of students in the upper group and the
lower group who select each incorrect alternatives. In addition, if the distracters
are not functioned, they should be rewritten or distracter.45
Harold s. Madsen stated in his book, there are three common causes of
weak distracter: fist, sometimes an item was drilled heavily in class, so almost
everyone has mastered the item, so the answer is obvious. Second, sometimes a
42
Asmawi Zaenul and Noehi Nasution, Penilaian Hasil Belaja,r, (Jakarta: UT, 2001),
p. 13.
43
Anthony J. Nitko, op. cit., p. 292.
44
Kathleen M. Bailey, Learning About Language Assessment, (Boston: ITP An
International Thomson Publishing Company, 1988), p. 130
45
Wilmar Tinambunan, Evaluation of Students …, p. 141.
25
well recognize pair is used, such as this/these, is/are, etc. even though not
everyone has controlled of these yet, and the third cause is the use of obviously
impossible distracter, for example: Did he o the work/ A. yes, he did, B. birds
eat worms, C. Train cannot fly.46
46
Harold S. Madsen, Techniques in …, p. 184
26
CHAPTER III
RESEARCH METHODOLOGY AND FINDING
A. Research Methodology
1. The Purpose of Study
The purpose of this research is to know the quality of English summative
test items at the second year students of SMPN 11 Depok seen from the level of
difficulty, discriminating power, and effectiveness of distracter.
2. Place and Time of Study
The research was held at SMPN 11 Depok that is located at Jl.
Murbai Komplek Sukatani Permai Depok. The writer did the research on
August 10th 2007, she took the test paper and the students’ answer sheet to be
analyzed.
3. Technique of Sample Taking
The writer took the sample from the second year students of SMPN 11
Depok, and the total number of the students is 375 students that are divided into
nine classes. But as a sample, she took only 85 students, they are taken from
two classes, they are class II.3 and II.5.
4. Technique of Data Collection.
To collect data based on the topic discussion, she did on observation by
visiting the school to ask for the students’ answer sheet and the paper of the
English summative test at the second year students of SMPN 11 Depok to be
analyzed.
5. Technique of data Analyzing
In this research, the writer used quantitative method to analyze the level of
difficulty, discriminating power, and the effectiveness of distracter of English
summative test items by using some statistic formulae, namely:
27
a. Level of Difficulty Analysis, by using the formula47
LD =
WL + WH
x 100%
nL + nH
Where LD
= Level of difficulty
WL
= The total number of wrong answer of the lower group
WH
= The total number of wrong answer of the high group
nL
= The number of the lower group
nH
= The number of the high group
With the classification as follows:
LD
= 25% - 75% = Good difficulty
25%
= Very easy item
75%
= Very difficult item
b. Discriminating Power Index by using the formula48:
DI =
WL + WH
n
Where DI = The discriminating Power Index
n = The number of students in the high group of lower group
With the classification as follows49:
DI =
0. 40
= Very good
0. 30 – 0. 39 = Good
0. 20 – 0. 29 = revised
0. 19
= Discarded
47
Wayan Nurkancana dan Sumartana, Evaluasi Pendidikan,(Surabaya: Usaha
NAsional,1986), P.136
48
49
Wayan Nurkancana dan Sumartana, Evaluasi…, P. 136
Mujijo, Tes Hasil Belajar, (Jakarta: Bumi Aksara, 1995), P.69
28
c. The effectiveness of distracter by using the formula50:
(H + L)
K=
D=
100%
nH + nL
25% x 1
(nH + nL)
2 x nD
Where K = Key answer
H
= The number of the high group who choose the option
L
= The number of the lower group who choose the option
nH = The number of the students in the lower group
nL = The number of students in the lower group
D
= Distracter
nD = The number of distracter
With the classification as follows:
Key answer = 25% - 75% / frequency H > frequency L = effective
< 25% / frequency H < frequency L = effective
> 75% / frequency H = frequency L = ineffective
Distracter
=
0. 6875 / frequency H < frequency L = effective
< 0. 675 / frequency H > frequency L = ineffective
B. Research Finding
1. Description of Data
The data that the writer used in this study is English summative test, which
is Ulangan Umum Semester Genap (II)51 for the second year students of SMPN
11 Depok academic 2007. The total numbers of test items are 45 test items
50
51
Wayan Murkancana, Evaluasi Pendidikan,h.143
See APPENDIX
29
which consist of 40 multiple choice test items and 5 essay test items52. The
multiple choice items consist of 21 reading items, 2 structure, 9 vocabulary
items, 2 general knowledge items, 11 language focus items. The test was held
on Monday, 14 June 2007 with the given time 90 minutes.
2. Data Analysis
No
1
2
3
4
5
6
7
8
9
10
Table 1
The Students scores of The English Summative Test Result for 95
Students of the Second Year Students of SMPN 11 Depok academic
year 2007
Score
27%
No
Score
46%
NO
SCORE
39
24
30
63
23
38
64
23
25
30
36
65
23
26
30
36
66
22
52
27
29
See APPENDIX
35
67
22
28
29
35
68
22
H
29
29
34
69
22
30
29
I
34
70
21
31
29
G
34
71
21
32
29
H
34
72
20
33
28
27%
L
O
W
E
30
The data that will be analyzed is 27% from the upper group and 27% from
the lower group and the middle group is ignored.
a. Level of Difficulty
1) Reading
31
a. Stated Detail Question
Table 2
Level of Difficulty of Stated Detail Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
13
30
65.21
Good Difficulty
1
0
1
2.17
Very Easy
3
4
0
4
8.69
Very Easy
4
20
2
22
47.82
Good Difficulty
5
3
0
3
6.52
Very Easy
7
11
0
11
23.91
Very Easy
18
6
1
7
15.21
Very Easy
20
17
13
30
65.21
Good Difficulty
21
2
1
3
6.52
Very Easy
22
3
0
3
6.52
Very Easy
24
1
0
1
2.17
Very Easy
25
1
0
1
2.17
Very Easy
26
0
0
0
0
Very Easy
31
19
7
26
56.52
Good Difficulty
32
21
4
25
54.34
Good Difficulty
Lower Group
High Group
(WL)
(HG)
1
17
2
Item
Based on the table above, we can see that in stated detail questions, there
are five items having good difficulty with level of Difficulty between 25% 27%, they are items no.1, 4, 20, 31, and 32. And the others are very easy
because the level of difficulty is under 25%, they are items no. 2, 3, 4, 5, 18, 21,
22, 24, 25, and 26.
b) Unstated Detail Questions
Table 3
Level of Difficulty of Unstated Detail Questions
32
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
23
12
8
WL+WH
LD%
Explanation
35
76.08
Very Difficulty
There is one unstated detail questions based on the data above that no. 8 is
very difficult that is 76.08%.
c) Pronoun Inferences
Table 4
Level of Difficulty of Pronoun Referents Questions
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
6
4
6
WL+WH
LD%
Explanation
10
21.73
Very Easy
There is one Pronoun Inferences based on the data above that is no. 6 with
have very easy that is 21.73%.
d) References
Table 5
Level of Difficulty of References Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
13
29
63.04
Good Difficulty
14
5
19
41.30
Good Difficulty
19
7
7
14
30.43
Good Difficulty
33
17
1
18
39.14
Good Difficulty
Lower Group
High Group
(WL)
(HG)
15
16
17
Item
There are four references based on the data above, they are no. 15, 17, 19,
and 33 and all of them have good difficulty, they are 63.04 %, 41.30%, 30.43%,
and 39.14%.
33
2). Structure
Table 6
Level of Difficulty of Structure Questions
No.
Item
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
3
3
38
WL+WH
LD%
Explanation
6
13.04
Very Easy
There is one structure question based on the data above no. 38 is very easy
that is 13.04%.
3). Vocabulary
Table 7
Level of Difficulty of Vocabulary Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
0
3
6.52
Very Easy
12
1
13
28.26
Good Difficulty
11
18
5
23
50
Very Easy
12
16
20
36
78.26
Very Difficulty
13
17
2
19
41.30
Very Easy
16
17
7
24
52.17
Good Difficulty
23
13
1
14
30.43
Good Difficulty
Lower Group
High Group
(WL)
(HG)
9
3
10
Item
In vocabulary there are seven items, we can see no. 12 is very difficult item
78.26%. And others are very easy level of difficulty. There are no. 9, 10, 11, 13
16, and 23.
4). General Knowledge
Table 8
Level of Difficulty of General Knowledge Questions
34
Total Wrong Answer
No.
Item
14
Lower Group
High Group
(WL)
(HG)
13
7
WL+WH
LD%
Explanation
20
43.47
Good Difficulty
Based on the data above, there is no. 14 has good difficult that is
43.47%.
5). Language Focus
Table 9
Level of Difficulty of Language Focus Questions
No.
Total Wrong Answer
WL+WH
LD%
Explanation
0
9
19.56
Very Easy
16
7
23
50
Very Easy
29
9
1
10
21.73
Very Easy
30
13
3
16
34.78
Good Difficulty
34
19
4
23
50
Good Difficulty
35
12
5
17
36.95
Good Difficulty
36
17
3
20
43.47
Good Difficulty
37
16
0
16
34.78
Good Difficulty
39
15
5
21
45.65
Good Difficulty
40
21
4
25
54.34
Good Difficulty
Lower Group
High Group
(WL)
(HG)
27
9
28
Item
There are ten language focus questions based on the data above, there are
three items are very easy in level of difficulty they are no. 27, 28, 29. And
others have good difficulty they are no. 30, 34, 35, 36, 39 and 40.
b. Discriminating Power Index
1) Reading
a. Stated Detail Question
35
Table 10
Discriminating Power Index of Stated Detail Questions
No.
Total Wrong Answer
WL+WH
DP%
Explanation
13
4
0.17
Discarded
1
0
1
0.04
Discarded
3
4
0
4
0.17
Discarded
4
20
2
18
0.78
Very Good
5
3
0
3
0.13
Discarded
7
11
0
11
0.47
Very Good
18
6
1
5
0.21
Revised
20
17
13
4
0.17
Discarded
21
2
1
1
0.04
Discarded
22
3
0
3
0.13
Discarded
24
1
0
1
0.04
Discarded
25
1
0
1
0.04
Discarded
26
0
0
0
0
Discarded
31
19
7
12
0.52
Very Good
32
21
4
7
0.30
Good
Lower Group
High Group
(WL)
(HG)
1
17
2
Item
From the above data, we can see there are 15 stated detail questions. Three
of them have very good discriminating index, and they are no. 3, 7, and 31 with
discriminating power between 0.47% until 0.78%. One of them has a good
discriminating power of is 0.30% that is no. 32 and one item has to be revised
that is no. 18 with 0.21%. And 10 items must be discarded because their
discriminating power are between 0 – 0.17%, they are items no. 1, 2, 3, 5, 20,
21, 22, 24, 25, and 26.
b. Unstated Detail Question
Table 11
Discriminating Power Index of Unstated Detail Questions
36
No.
Item
8
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
23
12
WL+WH
DP%
Explanation
11
0.47
Very Good
There is one unstated detail question has very good discriminating power,
that is no. 8 with discriminating power 0.57%.
c. Pronoun Referents
Table 12
Discriminating Power Index of Pronoun Referents
No.
Item
6
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
6
4
WL+WH
DP%
Explanation
2
0.08
Discarded
We can see there is one has to be discarded in Discriminating power of
Pronoun Referents that is no. 6 with discriminating power index 0.08%.
d. Inferences
Table 13
Discriminating Power Index of Inferences
No.
Total Wrong Answer
WL+WH
DP%
Explanation
13
3
0.13
Discarded
14
5
9
0.39
Good
19
7
7
0
0
Discarded
33
17
1
16
0.69
Very Good
Lower Group
High Group
(WL)
(HG)
15
16
17
Item
Based on the data above, there are four inferences. One of them has very
good discriminating power that is no. 33 with discriminating power 0.69%.
Moreover, one has good of discriminating power that is no. 17 with 0.39%. The
others two item that have to be discarded that is no. 15 and 19 with
discriminating power 0.13 and 0.
37
2) Structure
Table 14
Discriminating Power Index of Structure
No.
Item
38
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
3
3
WL+WH
DP%
Explanation
0
0
Discarded
From the above data, there is one structure in discriminating power index
that has to be discarded, that is no. 38 with discriminating power index 0%.
3) Vocabulary
Table 15
Discriminating Power Index of Vocabulary
No.
Total Wrong Answer
WL+WH
DP%
Explanation
0
3
0.13
Discarded
12
1
11
0.47
Very Good
11
18
5
13
0.56
Very Good
12
16
20
-4
-0.17
Discarded
13
17
2
15
0.65
Very Good
16
17
7
10
0.43
Very Good
23
13
1
12
0.52
Very Good
Lower Group
High Group
(WL)
(HG)
9
3
10
Item
Based on the data, we can see there are seven vocabularies. Two of them
have to be discarded that are no. 9 with 0.13% and no. 12 with -0.17%. But the
others have very good discriminating power, they are no. 10, 11, 13, 16 and 23
with discriminating power index from 0.43% – 0.65%.
4) General Language
Table 16
38
Discriminating Power Index of General Language
No.
Item
14
Total Wrong Answer
Lower Group
High Group
(WL)
(HG)
13
7
WL+WH
DP%
Explanation
9
0.39
Good
Based on the data above, there is one has good discriminating power index
of pronoun references that is no. 14 with 0.39%.
5) Language Focus
Table 17
Discriminating Power Index of Language Focus
No.
Total Wrong Answer
WL+WH
DP%
Explanation
0
9
0.39
Good
16
7
9
0.39
Good
29
9
1
8
0.34
Good
30
13
3
10
0.43
Very Good
34
19
4
15
0.65
Very Good
35
12
5
7
0.30
Good
36
17
3
14
0.60
Very Good
37
16
0
16
0.69
Very Good
39
15
5
9
0.39
Good
40
21
4
7
0.30
Good
Lower Group
High Group
(WL)
(HG)
27
9
28
Item
Based the data above, there are 10 language focuses. There are four items
having very good discriminating power index, the items are no. 30, 34, 36, and
37, with discriminating power index 0.43% - 0.69%. And the others are good;
they are no. 27, 28, 29, 35, 39, and 40, with discriminating power index 0.30%
- 0.39%.
39
c. The Effectiveness of Distracter
1). Reading
a. Stated Detail Question
Table 18
The Effectiveness of Distracter of Stated Detail Questions
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
1
A
3
7
Effective
B
9
8
Ineffective
C*
10
7
Effective
D
1
1
Ineffective
2
A
B*
Ineffective
23
22
C
3
5
Ineffective
D
1
Effective
A
1
Effective
B
1
Effective
19
Effective
D
2
Effective
A
1
Effective
C*
4
23
B
3
17
Effective
C
7
2
Ineffective
D*
13
3
Effective
A*
23
20
Effective
1
Effective
B
C
Ineffective
D
7
Effective
A*
B
23
1
Effective
12
Effective
1
Effective
40
18
C
7
Effective
D
3
Effective
A
1
5
Effective
B*
22
17
Effective
1
Effective
C
D
20
21
Ineffective
A
9
14
Effective
B
2
3
Effective
C
2
D*
10
Effective
6
Effective
A
1
Effective
B
1
Effective
21
Effective
C*
23
D
22
A*
Ineffective
23
20
B
Ineffective
C
24
3
Ineffective
A
Ineffective
23
22
C
1
A
Effective
Ineffective
B
C*
Effective
Ineffective
D
26
Effective
D
B*
25
Effective
23
1
Effective
22
Effective
D
Ineffective
A
Ineffective
B*
23
23
Ineffective
C
Ineffective
D
Ineffective
41
31
A
5
B
32
11
Effective
6
Effective
C*
16
4
Effective
D
2
2
Ineffective
A
1
9
Effective
B
1
3
Effective
C*
19
2
Effective
D
2
9
Effective
Based on the above, we can see there are 60 answers key from 15 the
answer key of stated detail question items they are 26 answer keys of the
effectiveness of distracters and 20 answer keys ineffectiveness of distracters.
b. Unstated Detail Question
Table 19
The Effectiveness of Distracter of Unstated Detail Questions
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
8
A
2
6
Effective
B
2
8
Effective
C
8
9
Effective
D*
11
Effective
The table above shows all of the answer keys are effectiveness of
distracters.
c. Pronoun Referents
Table 20
The Effectiveness of Distracter of Pronoun Referents Questions
No. Item
Option
6
A
High Group (H)
Lower Group (L)
Explanation
Ineffective
B*
19
17
Effective
C
1
5
Effective
42
D
3
1
Effective
The table show there is one answer key in ineffectiveness of distracter and
two answer keys are effectiveness of distracters.
d. Inferences
Table 21
The Effectiveness of Distracter of Inferences Questions
No. Item
Option
High Group (H)
15
A
9
B
1
7
Effective
C
7
9
Effective
D*
6
7
Ineffective
A
2
8
Effective
B*
18
9
Effective
C
3
1
Ineffective
5
Effective
17
Lower Group (L)
Ineffective
D
19
33
Explanation
A
1
Ineffective
B
1
1
Ineffective
C*
16
16
Ineffective
D
5
6
Effective
8
Effective
A
B*
15
6
Effective
C
6
6
Ineffective
D
2
3
Effective
Based on the data above, we can see that there are 16 answer keys from 4
stated detail question items they are 7 the answer keys of effectiveness of
distracter and 5 answer keys of ineffectiveness of distracters.
2) Structure
Table 22
The Effectiveness of Distracter of Structure
43
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
38
A*
20
20
Ineffective
B
Ineffective
C
1
D
2
Effective
3
Effective
From the data above, we can see that there are two the answer keys of
effectiveness of distracters and one in ineffectiveness of distracter.
3) Vocabulary
Table 23
The Effectiveness of Distracter of Vocabulary
No. Item
Option
9
A
10
11
12
High Group (H)
Explanation
2
Effective
B
Ineffective
C
Ineffective
D*
23
21
Effective
A
22
5
Ineffective
B*
1
11
Effective
C
5
Effective
D
2
Effective
A
2
6
Effective
B
1
10
Effective
C
2
2
Ineffective
D*
18
5
Effective
A
1
7
Effective
B
19
6
Ineffective
2
Effective
7
Ineffective
14
Effective
2
Ineffective
C
D*
13
Lower Group (L)
3
A
B
2
44
C*
21
D
16
A*
16
B
23
6
Effective
1
Effective
6
Effective
8
Effective
C
3
7
Effective
D
4
2
Ineffective
11
Effective
A
B
Ineffective
C*
22
9
Effective
D
1
3
Effective
Based on the data above, there are 28 answer keys from 7 vocabulary items,
they are 13 answer keys of effectiveness of distracters and 8 answer keys of
ineffectiveness of distracters.
4) General Knowledge
Table 24
The Effectiveness of Distracter of General Knowledge
No. Item
Option
High Group (H)
Lower Group (L)
Explanation
14
A
7
10
Effective
B
1
Effective
C
2
Effective
10
Effective
D*
16
We can see that all of the answer keys are effectiveness of distracters.
5) Language Focus
Table 25
The Effectiveness of Distracter of Language Focus
No. Item
Option
27
Lower Group (L)
Explanation
A
3
Effective
B
3
Effective
14
Effective
C*
High Group (H)
23
45
D
28
2
5
Effective
B*
16
7
Effective
4
Effective
D
5
7
Effective
A*
22
14
Effective
1
Effective
7
Effective
1
Effective
11
Effective
B
1