T1 112005156 Full text
The Perceptions of English Department Students on the Use of
Multiple-choice Questions
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Merry Tan
112005156
ENGLISH DEPARTMENT
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2013
The Perceptions of English Department Students on the Use of
Multiple-choice Questions
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Merry Tan
112005156
ENGLISH DEPARTMENT
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2013
i
COPYRIGHT STATEMENT
This thesis contains no such material as has been submitted for examination in any
course or accepted for the fulfillment of any degree or diploma in any university. To
the best of my knowledge and my belief, this contains no material previously
published or written by any other person except where due reference is made in the
text.
Copyright@ 2013 Merry Tan and Dra. Martha Nandari S. Handoko, MA.
All rights reserved. No part of this thesis may be reproduced by any means without
the permission of at least one of the copyright owners or the English Department,
Faculty of Language and Literature, Satya Wacana University, Salatiga.
Merry Tan:
iii
Introduction
There are pros and cons perceptions in the society about multiple-choice
test questions (MCQs). Many people think that MCQs are not quite effective for the
educational world because it is hard to measure students‟ ability level. However,
MCQs are the most popular question type used in teaching and learning process
(Oosterhof, 2001, Tozoglu et al., 2004). According to Lee (2011) as cited in Ihsan
Ibadurrahman, MCQs are the most commonly used testing type in many academic
achievement tests but MCQs are also the most criticized type. Some students might
be more able to perform on MCQs better than other question types. Generally,
students' perceptions or attitudes toward the types of question could differ according
to their outlooks about their performance on the test. Moreover, although most of the
students report that multiple-choice questions are easier than the other question
types; they could consider multiple-choice questions are more difficult than the other
question types in some situations (Holtzman, 2008). The explanation of those results
might connect with students‟ belief about the process of preparing for this type of
questions. So, they perhaps think that with objective questions, it is easier to study
the material. Zeidner (1987) stated that students' perceptions' toward test types can
differ in terms of different works, such as how they perceived interestingness,
usefulness, trickiness, fairness and other elements.
Multiple-choice questions also have advantages and disadvantages in
teaching and learning process. It is supported by the statement from Ben-Simon,
Budescu and Nevo, (1997) that stated “Multiple-choice tests also have advantages
which are mentioned that they easy to score, can be described objective in terms of
their scoring, easy to computerize specially in term of scoring process, and more
importantly it is easy to cover the objectives of the course through this type of
1
exams.”. Overall, there are many problems that appeared from this type of questions,
such as the guessing issue, the difficulty to measure the real students‟ ability, and the
process of preparing questions which is not easy because it needs time and efforts
(Oosterhof, 2001). Some researchers still argued that this type of questions can
measure the complexity of academic processes effectively (Mehrens and lehman,
1984). The literature indicates that students prefer this type of exam comparing with
other types exam (Zeidner, 1987; Choi, 1998; Birenbaum and Feldman, 1998). The
details of these results might linked with students‟ perceptions about the process of
preparing for this type of exam, as the result, they perhaps believe that with objective
exams it is easier to study the material, and they might think that multiple-choice
type question is more fair than the other types question.
Zeidner (1987) tried to study secondary school students' perceptions of
essay and multiple choice type questions, the participants of his study consisted of
174 male and female students. Two instruments were applied in this study. First, he
represented students' perceptions in terms of ten scopes: difficulty, complexity,
clarity, interest, trickiness, fairness, value, success expectancy, degree of anxiety
evoked, and feeling at ease. The second instrument was used as standard instrument,
to observe the validity of the major instrument, this instrument included seven
scopes. In the second tool the students were asked to answer whether they prefer
essay or multiple choice type questions. The results of this study revealed that
secondary school students preferred multiple-choice question more than essay
question.
Furthermore, Nassar (2006) investigated the Hashemite University students‟
training on skills related to their achievement on true-false, multiple choice,
completion, and essay questions. He found that the training was positive in
2
developing students' skills in dealing with the mentioned questions. In general,
Nassar observed that the biggest problem that students dealt with was obtained in
students' achievement on essay questions. He explained that this result due to
students' perceptions of essay questions before and after the training. As the result,
students might have negative perceives toward this type of questions because they do
not have sufficient qualifications of dealing with such questions.
The study of English Department (ED) students‟ perceptions about the use
of multiple-choice type questions on the learning process is being focused to
determine the beneficial and harmful effects on doing the MCQs. This study surely
has a major effect to establish and to recognize the important of multiple-choice
questions. The project‟s goal is designed to help students improve their achievement
level, to immobilize the misunderstanding toward multiple-choice questions, and to
decide the validity of multiple-choice test questions for sure. This goal can be
achieved through the teacher who can teach the students the tricks on doing the
multiple-choice test questions and English Department/ED students‟ help to talk
about their perceptions toward the use of multiple-choice test questions on the
learning process. The output of this study is various perceptions about the use of
multiple-choice test questions on the learning process that can make the students
think about the benefits and harmful of multiple-choice test questions and
disseminate by renewal the negative perceptions to positive perceptions.
The research questions to be answered are:
1. How do the English Department (ED) students think about the use of
multiple-choice questions in their tests?
3
2. According to the English Department (ED) students, what are the
beneficial effects and the harmful effects of multiple-choice questions to
their learning process?
Multiple-choice questions are basically recognition tasks because students
must identify the correct response. The multiple-choice questions are supposed to be
a very flexible part of assessment format that are used to measure knowledge, skills,
abilities, values, thinking skills, etc (Haladyna, 1999; McDougall, 1997). Besides
that, a test usually consists of a number of questions and the students must choose
the right answer among the number of choices. Therefore, students are trained to use
their brain to think logically. Then, multiple Choice Questions (MCQs), also called
as an objective test, is one of the test types in language assessment where
participants are typically asked to respond by giving one correct answer from the
available options (Hughes, 2003). Hughes (2003, pp.76-78) stated that multiplechoice questions are all receptive or selective, response items chosen by the testtaker from a set of responses or usually called as a supply type of response rather
than creating a response. Multiple-choice questions have a stem, which includes a
stimulus, and options usually between three and five to choose. One of the options is
the correct response, while the others are the distractors.
As we know, there are many beneficial effects of the use of multiple-choice
questions on teaching and learning process. According to Haladyna (1999), multiplechoice questions are considered to be beneficially an efficient and effective way to
assess a wide range of knowledge, skills, attitudes and abilities. Multiple-choice tests
can also be a useful tool to develop students‟ thinking skill (Haladyna, 1999).
According to Hughes (2003, pp. 76-78), multiple-choice questions are
practical and reliable. With their programmed correct responses and time-saving
4
scoring procedures, it is easier for the teacher on grading and scoring. It is also easier
for the students to answer the MCQs because in each question, there is a correct
answer in between the options for sure.
Generally people have the opinion that MCQs only deal with lower-order
analytical skills that require knowledge, application and comprehension only, but not
the high-order skills including critical thinking and problem solving skills.
According to William (1996) & Mehrens (1998), the latest test appliances spending
multiple-choice questions to test higher order analytical skills or which include
analysis, synthesis, and evaluation to display students‟ point. Higher order analytical
skills make the limited errors in doing multiple-choice questions. To develop these
skills, students can debate, write and master the structured argument. Smothering
education is needed but of course with high responsibility. The example of this
analytical skill is students have to collect the data by using the internet. McGuinness
(1999) points out that higher order analytical skill includes categorizing and
analyzing information, drawing conclusions from the information, „brainstorming‟
new ideas, and solving the problem. Besides that, he also added that higher order
analytical skill includes determining cause and effect, evaluating options, planning
and setting goals, monitoring progress, making decision, and reflecting on one‟s own
progress.
William (1996) & Mehrens (1998) mentioned further that people have
discarded the statements that multiple-choice format is competent of dealing with
lower order skills of recall or recognition. Lower order skills are used if the learner
does not validate, question, or evaluate, and in the end the information obtained. The
example of this skill is when students have to collect the information by using
5
electronic textbook. According to McGuinness (1999) collecting information is the
lower order analytical skills.
Although some people think that multiple-choice questions are effective for
the teaching and learning process, actually there are also many harmful effects that
students and teacher faced from the use of multiple-choice tests. Some people claim
that multiple-choice questions are too easy for the students. It makes the students
lazy to study. Multiple-choice questions are easiest to write when there is a
definitively right or wrong answer (Bloom, Benjamin B, 1956).
The bad effect of multiple-choice questions that clearly appear on the
learning process is the guessing issue. The score is unpredictable because this is just
depending on the luck. However, the students would usually prefer multiple-choice
questions than the essay or other question types because of that reason. Students
think that MCQs are helpful when they forgot the material because they can guess
the answer. According to Espinosa & Gardeazabal (2009), students are motivated to
have a high possibility to guess in doing multiple-choice questions. Besides that,
candidates may be able to pick the correct answer without having to read the passage
or understanding the meaning in it (Lee, 2011: 31). Similarly, a correct answer can
also be the result of eliminating incorrect answers, without the candidates knowing
the right answer in the first place (Woodford & Bancroft, 2005).
People think that multiple-choice questions are always objective and highly
reliable for most students to improve the skills, but this makes the students idle in the
learning process. They just have to choose one of the options that they think is the
true one whereas we don‟t know if they understand the lesson or not. Related to the
learning skills‟ disability, people argued that students can improve their marks on
multiple choice tests but practice in taking multiple-choice questions and test-taking
6
strategies will not give learners with the most effective ways of improving their
language ability (Hughes, 2003).
Multiple-choice questions are time consuming. Just like Burton, Sudweeks,
Merrill, & Wood (1991) argued that good multiple-choice questions are more
difficult and more wasting the time to make than other types of test questions that
require skills. This skill, however, may be increased through study, practice, and
experience.
Multiple-choice questions are known as a simplest kind of question types to
create, but it is extremely difficult to design appropriately. Hughes (2003, pp. 76-78)
mentioned that the weaknesses of multiple-choice questions are only technique tests
that recognize knowledge and severely limit what can be tested. It mentioned that
guessing and cheating may have a significant effect on test scores.
Adapted from Gronlund (1998, pp. 60-75) and J.D. Brown (1996, pp. 5457), there are four guidelines for designing multiple-choice questions for both
classroom-based and large-scale situations. The first one is designing each item to
measure a specific objective (the comprehension of wh-questions. Next is stating
both stem and options as simply and directly as possible. To make it as a good
multiple-choice question, it‟s better to make it not too wordy and get directly to the
point. The third one is being sure that the intended answer is clearly the only correct
one. Then, the last guideline for designing multiple-choice items is using item
indices to accept, discard, or revise items.
Eliminating not premeditated possible answers is often the most difficult
problem of handling multiple-choice items. According to Gronlund (1998, pp. 6075) and J.D. Brown (1996, pp. 54-57), There are three indices that appeared on
measuring items to choose the proper selection and suitable multiple-choice items on
7
a test. First is item facility (IF). Item facility is the scope to which an item is easy or
difficult for the planned group of test-takers. The teacher should classify the
questions carefully to make it not too difficult and too easy because it‟s invalid to
separate the high- and low-ability test-takers. Second is item discrimination (ID).
Item discrimination is the scope to which an item differentiates between high- and
low-ability test takers. Poor ID would appear when the high- and low-ability students
score equally well because it did not discriminate between the two groups. The last
index is distractor efficiency which is related to item discrimination. The example is
in a test at least two distractors attracting some attention from lower ability test
takers.
The Study
In order to answer the questions of this study, the quantitative method,
namely survey research, was used. Then, to convince and analyze my own
perspective of my study, I used qualitative descriptive method with an open-ended
style. Open-ended (or “open question”) is a question where the researcher doesn‟t
provide the respondent with a set answer from which to choose. Rather, the
respondent is asked to answer "in their own words". In this case, I used an essay
question to provide students‟ perspective. The participants of this study were
seventy-four students from basic grammar classes in the English Department of
Satya Wacana Christian University in the second semester of the academic year
2012-2013.
Data Collection Instrument
I developed a questionnaire as the instrument to collect the data. This
instrument consisted of two sections. The first section consisted of the fifteen
8
questions which the students had to choose for the best answer, and the second
section consisted of an essay question which asked about students‟ perceptions of the
use of MCQs in their tests. The questions of perceptions‟ were prepared by the
researcher and the ideas of these questions were taken from some researchers‟
perceptions toward the use of MCQs. The first three questions of the first section
were made to find out the students‟ perceptions on the difficulty and possibility of
guessing, cheating, and ambiguity of MCQs. The same procedure was developed
with the questions which were constructed to measure the students' skills by using
MCQs on their tests. Next parts of the questions were developed to find out students‟
perceptions toward the benefits of the use of MCQs. The instrument was constructed
as a Likert-type rating scale on a four-point range that was scored and categorized as
following: Strongly Agree=4, Fairly Agree=3, Fairly Disagree=2, Strongly
Disagree=1. Higher scores on the questions indicate more favorable dispositions
toward the use of multiple-choice questions type under consideration. Finally, the
writer counted the results one by one each questions.
Data Collection Procedures
In the beginning, the students from two Basic Grammar classes (thirtyseven students of each class) were selected randomly. The number of all of the
students who were in those classes was seventy-four. The questionnaire was
administered to all of the students selected for this study. The students from one of
the selected Basic Grammar class were participated all because no absent student at
that time, however one of the Basic Grammar class which participated in this study
was selected randomly from the rest three classes of Basic Grammar class students.
They were informed about the list of beneficial and harmful effects of multiplechoice questions before responding to the questionnaire. Students were expected to
9
choose whether they agreed or disagreed with the list of beneficial and harmful
effects of MCQs. Students respond to the instrument with the name and student ID
written on, and they need in average 5-10 minutes to accomplish the task. The
process of collecting the data most of the time was done by the researcher or with the
help of friends or some teachers who were teaching in the selected classes.
Quality of Data
For the purpose of examining the validity of the instrument (face validity
evidences), I used SPSS to make sure for the validity. After inputted the data to
SPSS, I used the descriptive reliable to measure it. After got the reliability statistics
and item-total statistics, the questions could be determined the validity and
reliability. If the value of Cronbach‟s Alpha was > 0.70, it means that the test was
reliable. And if the value of corrected item-total correlation was > r table, it means
that the test was valid.
Data Analysis Procedure
Besides that, an essay question was made to find out if there were other
opinions about the use of MCQs to support the answers in the first section (list of
questions). Sample questions from the instrument are as follow: "It is easier to get
high score in MCQs", "MCQs are practical and reliable", "MCQs decrease the
anxiety", "and MCQs differentiate good, average, and weak students". In order to
answer the research questions, I explained questions with the dominant answers one
by one with the supporting statements from the essay question.
Discussion
In order to describe the English Department students‟ perceptions toward
the use of multiple-choice questions, there were fifteen questions given to them.
10
Those questions were about the advantages and disadvantages of multiple-choice
questions.
Table 1. MCQs increase the chance of guessing.
SA
FA
FD
13
32
SA = Strongly Agree
19
SD
10
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As shown in the first question, there were thirty-two participants who fairly
agreed that multiple-choice questions increased the chance of guessing. Some of
them thought that the chance of guessing was the good idea to raise their mark and to
help when they forgot the material. Aprilia L, one of the students in the class of basic
grammar, said that “If we don’t have any idea about the answer, we can choose one
of the choices”. Not much different with Aprilia L, Laurensia Pamela assumed that
“If we really don’t know the answer, we can guess the answer and if we are lucky we
ca n get the great score”. That means those students perceived the positive points of
guessing. Multiple-choice assessments with a high number of stems and options
helped the students to get at least 25% if the test-takers picked one of the options
continuously by purely guessing. Intan Ariani argued that “Multiple-choice
questions give a chance for the students to guess if they really don’t know the
answer. It is really helpful, but on the other hand, students become lazy to study
because they still have at least 25% chance to guess the right answer.”
Table 2. MCQs reduce the chance of cheating.
SA
FA
FD
5
18
SA = Strongly Agree
28
SD
23
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
When asked whether multiple-choice questions reduce the chance of
cheating, it was revealed that the participants in this research fairly agreed but not a
11
few who disagreed with the question. Some students perceived that “Multiple-choice
questions will increase the chance of cheating.” It had been spreading over the world
that multiple-choice questions could not be separated with the issue of cheating.
Yayas K.S. also said that “multiple-choice will increase the chance of cheating with
other students.”
Table 3. MCQs have more options that help the students in comparison and increase
the ambiguity.
SA
FA
FD
SD
29
32
SA = Strongly Agree
9
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Towards question number three “multiple-choice questions have more
options that help students in comparison and increase the ambiguity” students
strongly agreed. Many of them also argued that “multiple-choice questions are really
confusing and so ambiguous because the answer is quite similar and it’s trapping.”
Actually, the confusing part was located in the question construction. Focusing on
important content was not enough.
Table 4. MCQs improve students‟ language skills.
SA
FA
FD
6
37
SA = Strongly Agree
25
SD
6
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As predicted, the students fairly agreed with the fourth question which
asked if multiple-choice can improve students‟ language skills. One of the students
argued that “we can increase our ability with new vocabulary.” Although it was
difficult to increase the students‟ language ability with multiple-choice questions,
actually it was simpler when the students measured their language ability.
Table 5. MCQs develop students‟ thinking skills.
12
SA
FA
7
47
SA = Strongly Agree
FD
18
SD
2
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Then, as shown in the fifth question, the participants showed their interest
and agreement in this question (multiple-choice questions develop students‟ thinking
skills). Kartika Dianing R wrote that “Multiple-choice questions can develop
students’ thinking skills.” Similar to Kartika, Raissa Junita also said that “Multiplechoice questions can help her thinking skills.” There were many positive opinions
about developing students‟ thinking skills.
Table 6. MCQs make students save time in doing the test.
SA
FA
FD
SD
27
34
SA = Strongly Agree
12
1
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Concerning question number six about time-saving in doing the test, the
students strongly agreed with that idea. Nadia Angela told that “We can save a lot of
time to check our answer again.” Not only Nadia who argued that multiple-choice
questions make students save time in doing the test, but also Santa Maya Pramusita
said that “The benefit of multiple-choice questions is we don’t have to write too
much, so it can save the time.”
Table 7. MCQs encourage the students to study.
SA
FA
FD
3
37
SA = Strongly Agree
31
SD
3
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Next focusing on question number seven about multiple-choice questions
that can encourage the students to study, again the students fairly agreed. Chang Sese
assumed that “Multiple-choice sounds like a riddle that makes me curious with the
13
answer, so it motivates me to study harder. I just need to take notes and study the
important points for my strategies.” However, there were some statements from the
participants that said “Multiple-choice makes students lazy to study.”
Table 8. MCQs are easier to get the high score.
SA
FA
FD
15
39
SA = Strongly Agree
19
SD
1
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
About the question number eight, the participants fairly agreed that
multiple-choice questions enabled them to get the high score easier. Rizza Faosiah
said that “Multiple-choice questions help students to easier get the high point.”
Same with Rizza, Laurensia Pamela also said that “If we are lucky we can get the
great score.” In fact, multiple-choice questions were easier to get score because
there were more options that made students easier to do the test.
Table 9. MCQs measure the real students‟ ability.
SA
FA
FD
2
24
SA = Strongly Agree
39
SD
9
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Dealing with question number nine about measuring the real students‟
ability, the participants also fairly agreed. Naomi Tade Djo agreed that “Multiplechoice questions can measure her real ability because she can learn about problem-
solving.” The ability to find out and to solve the stressed problem was more
important.
Table 10. MCQs are practical and reliable.
SA
FA
FD
10
SA = Strongly Agree
47
13
SD
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
14
As shown in the next question that stated that multiple-choice questions are
practical and reliable, the participants agreed. Anissa Putri argued that “Multiplechoice is a simpler way to answer the questions.” Another similar opinion came
from Jap Victoria, who said that “MCQs are quite helpful for students in the test
because it is easier compared to essay.”
Table 11. MCQs are valid.
SA
FA
12
44
SA = Strongly Agree
FD
14
SD
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
The next statement was about multiple-choice questions that were valid.
The participants fairly agreed with. Most of the students answered that “Multiplechoice questions are valid.”
Table 12. MCQs are objective.
SA
FA
17
46
SA = Strongly Agree
FD
11
SD
0
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As expected, in question number twelve, “Multiple-choice questions called
as an objective”, it was revealed that the participants in this research totally agreed.
The students agreed that “Multiple-choice questions are helpful for the students. It’s
also easier to do than the other type of questions.” It could not be denied that
multiple-choice questions were so helpful especially for the students and the teacher
too.
Table 13. MCQs are easy to do.
SA
FA
FD
15
SD
18
38
SA = Strongly Agree
14
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
There were a lot of the participants who agreed that multiple-choice were
easy to do. It was just like what Jessica said, “In my opinion multiple-choice makes it
easier for the students to answer .” Almost similar with what Jessica said, Dara also
argued that “It’s easier to do the exercise with multiple-choice because we just can
learn the important point, but for essay questions we have to think harder.” It means
that multiple-choice questions type is easier than the other question types.
Table 14. MCQs decrease the anxiety.
SA
FA
FD
9
50
SA = Strongly Agree
12
SD
3
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
The result of question number fourteen which asked if multiple-choice
questions decrease the anxiety was agreed. La Valeroe Novianne agreed that “MCQs
decrease the anxiety.” Similar with the students, Snow (1993) said that “MCQs
reduce students’ anxiety.”
Table 15. MCQs differentiate good, average, and weak students.
SA
FA
FD
SD
6
26
SA = Strongly Agree
33
9
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Last question was the very crucial question. The participants were asked
about MCQs that could differentiate good, average, and week students, and they
fairly agreed. Luky Rias Pambudi argued that “The discrimination of good, average,
and week students happened sometimes, because sometimes also good, average, and
week students get the similar score that make it harder for the teacher to
differentiate the ability.”
16
As expected, almost all of the students preferred multiple-choice questions,
even for the higher-achiever students. It seems that students have built their thought
that multiple-choice is easier to do because it has more options which are directed to
the real answer. Personally, I agree with my participants‟ perceptions and I think that
my participants weren‟t only copy paste from my statements only, because they
made their own statement and there was a statement that made me realize with the
fact that “always pick answer C and you‟ll be right about 25% of the time”. As we
know in general, it is impossible because the teacher should make the questions more
complex and unpredictable in scoring. However, the researches convince that it is
possible and even it is a must that at least 25% of the score will be given if the
students choose “C” all of the time. It is very interesting because it will happen if the
teachers are competent in the process of questions‟ making.
Conclusion and Suggestions
My study surely was made to establish and to recognize the important of
multiple-choice questions. In this study, students‟ perceptions on the use of multiplechoice questions were generally positive. Out of the four-point range, the students
chose strongly agree and fairly agree, instead of strongly disagree and fairly disagree
each question. As founded in this study, the students preferred multiple-choice
question type than the other question types such as essay question or fill in the blank
question type. Shockingly, there are some contradictory findings in this study.
Almost a half of the participants agreed that MCQs reduce the chance of guessing
and cheating. Like what we discussed before, half of them agreed that MCQs
increase the chance of guessing and cheating, but more of them chose to disagree
with it. Some revealed that there were no relation between MCQs and the chance of
17
guessing and cheating. Some stated that although there were options on the question,
if they chose the wrong answer, the point was obviously zero. As a conclusion, it
doesn‟t mean that if the students choose to guess and cheat in the test, they will get at
least 25% of the score no matter of what. The previous result might indicate that the
students perceive multiple-choice as a positive and the beneficial one. Although
there were pros and cons about multiple-choice around the participants, most of them
agreed that multiple-choice is very helpful. It is very obvious either through the
results of this study or the results of the other studies in literature that students in
general have positive thoughts toward multiple-choice questions. Surprisingly,
students fairly agreed with the statement “MCQs measure the real students‟ ability”
but heed that MCQs increase the chance of guessing. It doesn‟t mean that students
indicate guessing as a part of real students‟ ability, but sometimes not only real
students‟ ability that needed in MCQs but also the sense of proper guessing should
be needed. It proves that even the high-achievers also try to guess sometimes. The
more interesting result was that although students preferred multiple-choice
questions more than the other question types. They thought that multiple-choice
questions were the most confusing question type.
Implication
To make sure that students succeed in their multiple-choice questions exam,
the teachers should motivate the students to study harder when they have exam with
multiple-choice questions. It can improve students‟ thinking skills and language
skills with deep learning.
Suggestions
Due to the results of the current study and to mentioned results in the
literature, the following recommendations could be presented:
18
1. The students cannot underestimate the multiple-choice questions
because the more options means the more confusing they can be.
2. The teachers should make a report about students‟ perceptions of
multiple-choice questions. Since students' scores on this question type
is used to take decision about their real ability.
3. The teachers must be careful in making the good multiple-choice
questions. So that they are valid and reduce the ambiguity.
Limitation
The participants of this study were Basic Grammar students' of the first
grade in English Department of Satya Wacana Christian University of the academic
year 2012-2013. Also the results of this study should be interpreted in the context of
its definitions of the variables, its instrument, and the research method used to
answer the research questions.
Further Research
Other researchers could describe the beneficial and harmful effects of other
question types in the next research. They also could compare multiple-choice
question type with the other question types.
Acknowledgement
Foremost, I would like to express my sincere gratitude to my advisor Bu
Martha Nandari for the continuous support of my study and research, for her
patience, motivation, enthusiasm, and enormous knowledge. Her guidance helped
me in all the time of research and writing of this thesis. I could not have imagined
having a better advisor and mentor for my study. I also would like to thank my
second reader, Bu Maria Eko for her advices, suggestions, and tolerance for the ease
of my work.
19
Besides my advisor, I would like to show my biggest gratitude to best of
the best advisor of my life, God, You help me pass all of the difficulties I faced.
Please bless me for the next steps of my life too. Next, I would like to thank my
family: my lovely parents for the praying and the words that motivate me to pass this
study. Thanks to my cousins who help me a lot: Didi, Aweng, Chang Sese, Jessica,
Silvia, Agnes, Jenni, Weni, Hia Rein, Mengkhoi hia, Sok Pheng hia, LingCe, Cehu,
EngCe, Aso, etc. For my aunts and uncles who support me. I love you all.
I would like to thank my best friends: Alce and Izni for sharing the
difficulty and laugh together. Thank to my forever friends: TuaCe, TuaHia, Apang,
Bos, Hendi, Angie, Johny, Oni, and Wendi for filling my beautiful life. I always love
you. Wish you all the best and joy. Last, for my boarding house‟s friends, Maria,
Ani, Cepy, Mbak Ita, Pinot, Desi, Uning, Rosi, Marlyan, Wicha, Hery, Susy, Ita,
Jessica, etc, thank for everything. I will never forget you girls. Thanks for the
support, the joy, and the help when I needed something. Go… Go… Dipo 74A.
20
References:
Aiken, L. R. (2000). Psychological Testing and Assessment (10thEdition). Boston,
MA: Allyn and Bacon.
Bachman, L. & Palmer, A. (1996). Designing and developing useful language tests.
Oxford: Oxford University Press.
Ben-Simon, A., Budesco, D.V. and Nevo, B. 1997. A comparative study of measures
of partial knowledge in multiple-choice tests. Applied Psychological
Measurement, 21 (1): 65-88.
Biggs, J. B. (1973). Study behavior performance in objective and essay formats.
Australian Journal of Education, 17, 157-167.
Brown, J.D. (1996). Testing in language programs. Upper Saddle River, NJ:
Prentice Hall Regents.
Bush, M. (2001). A multiple-choice test that rewards partial knowledge. Journal of
Further and Higher Education, 25 (2), 157-163.
Cannon, R.A. and Newble, D. (1983) A Handbook for Clinical Teachers, Lancaster,
MTP Boston: p 97-105.
Carey, J. (1997). Everyone knows that E = MC2, who can explain it? Business Week,
3547, 66-68.
Chatterji, M. (2003). Designing and Using Tools for Educational Assessment.
Boston, MA: Allyn and Bacon.
Dufresene, R. J., Leonard, W.J., & Gerace, W.J. (2002). Making sense of students‟
answers to multiple-choice questions. The Physics Teacher, 40, 174-180.
Gronlund, N. E. (1998). Assessment of Student Achievement. Sixth Edition. Boston,
MA: Allyn and Bacon.
Holder, W. W., & Mills, C. N. (2001). Pencils down, computer up: The new CPA
exam. Journal of Accountancy, 191 (3), 57-60.
Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge:
Cambridge University Press.
Johnson, D. W. & Johnson, R. T. (2002). Meaningful Assessment: A Manageable
and Cooperative Process. Boston, MA: Allyn and Bacon.
Kreig, R.G., and Uyar, B. (2001). Student performance in business and economics
statistics: Does exam structure matter? Journal of Economics and Finance, 25
(2), 229-241.
Lee, J. (2011). Second Language Reading Topic Familiarity and Test Score: TestTaking Strategies for Multiple-Choice Comprehension Questions (PhD.
Thesis). Retrieved from ProQuest Dissertations and Theses. (Accession Order
No. 3494064).
Marshall, J. C., & Hales, L. W. (1971). Classroom test construction. Reading MA:
Addison-Wesley
http://www.utexas.edu/academic/mec/research/pdf/writingmcexamshandout.pd
f
21
McKeachie, W. J. (1999). Teaching Tips: Strategies, Research, and Theory for
College and University Teachers (10thEdition). Boston, MA: Houghton Mifflin
Company.
McTighe, J. & Wiggins, G. (2005). Understanding by design (2nd ed). Alexandria,
VA: Association for Supervision and Curriculum Development.
Popham, W. J. (2000). Modern Educational Measurement: Practical Guidelines for
Educational Leaders (3rdEdition). Boston, MA: Allyn and Bacon.
Stoynoff, S. & Chapelle, C. (2005). ESOL tests and testing. Alexandria, VA:
TESOL.
Snow, R. E. (1993). Construct validity and constructed-response tests. In E. B.
Randy, & C. W. William (Eds.), Construction versus choice in cognitive
measurement. Hillsdale, NJ: Lawrence Erlbaum, 45-60.
Trice, A. D. (2000). A Handbook of Classroom Assessment. New York: Addison
Wesley Longman, Inc.
Zeidner, M. (1987). Essay versus multiple-choice type classroom exams: The
student‟s perspective. Journal of Education Research, 80 (6), (July/August),
352-358.
22
Appendix A
Questionnaire:
I am doing a research about the students’ perceptions on the use of multiple-choice questions on the learning process. I would like to
know your opinions about the beneficial and harmful effects of multiple-choice questions.
INSTRUCTIONS: Please rate how strongly you agree or disagree with each of the following statements by placing a circle in the
appropriate box. Choose “strongly agree”, “fairly agree”, “fairly disagree”, or “strongly disagree” based on your opinion.
1. Multiple-choice questions reduce the chance of guessing. Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
2. Multiple-choice questions reduce the chance of cheating.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
3. Multiple-choice questions have more options that help
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
the students in comparison and reduce the ambiguity.
4. Multiple-choice questions improve students‟ language
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
skills.
5. Multiple-choice questions develop students‟ thinking
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
skills.
6. Multiple-choice questions make students save time in
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
doing the test.
7. Multiple-choice questions encourage the students to
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
study.
8. Multiple-choice questions are easier to get the high score. Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
9. Multiple-choice questions measure the real students‟
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
ability.
10. Multiple-choice questions are practical and reliable.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
11. Multiple-choice questions are valid.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
12. Multiple-choice questions are objective.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
13. Multiple-choice questions are easy to do.
14. Multiple-choice questions decrease the anxiety.
15. Multiple-choice questions differentiate good, average,
and weak students.
What do you think about the beneficial and harmful effects of multiple-choice questions on your learning process? Please give your
opinion about it and specify if you have the other ideas about characteristic of multiple-choice questions.
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………
Appendix B
Strongly disagree = 1
Fairly disagree = 2
Fairly agree = 3
Strongly agree = 4
Name
Andika W.
Handini S.
Wiwid Wahyuni
Ivana Christina H.
Venny C.R.
La Valeroe Novianne, Lie
Nadia Angela
Sabati Sandy Simonne
Sola Gratia Deo Wiyana
Marcellina A.G.H
Permata Dipaning Ratri
NIM
11200814
6
11201012
5
11201200
3
11201200
5
11201200
6
11201200
8
11201200
9
11201201
1
11201201
2
11201201
3
11201201
6
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
3
2
1
2
2
3
1
2
2
2
3
3
3
3
2
3
1
3
2
3
4
4
4
1
3
4
3
3
3
1
3
2
3
3
4
4
4
2
2
3
3
3
3
3
1
3
2
3
2
3
3
3
3
3
3
3
3
3
3
2
3
3
4
3
3
3
3
3
3
4
4
3
3
4
3
2
2
3
2
3
3
3
2
2
3
2
3
2
3
2
3
3
3
4
4
4
3
3
3
3
3
3
4
3
3
3
2
4
1
2
4
2
2
1
4
4
4
3
3
2
4
1
4
2
2
4
2
4
1
3
2
3
4
3
2
3
2
4
3
3
3
3
4
3
3
4
4
4
3
2
2
2
3
3
3
2
2
2
2
2
3
2
2
2
3
Joannita Ratih
Wiharmastu
Winda Adelia
Aprilia I
Vania Hendra Gunawan
Lidya
Y. Yubilate. R.
Belinda Gita Puspita
Nova Suciyanti
Jessica
Firsta P.S.
Intan Ariani
Windi Ismitasari
Grace Kartika R.
Sandra Priskila
Yoas Betty K.
Fajarini Prihastuti
11201201
8
11201201
9
11201202
1
11201202
4
11201202
5
11201202
6
11201202
7
11201202
8
11201202
9
11201203
0
11201203
1
11201203
2
11201203
4
11201203
6
11201203
7
11201204
1
2
1
4
4
4
4
2
2
1
2
3
4
3
1
4
3
3
3
3
3
3
3
3
2
3
4
3
4
3
4
3
2
3
3
3
4
3
3
2
3
3
3
3
2
2
4
4
3
2
3
3
2
2
3
3
3
3
3
2
3
2
2
3
3
2
2
3
2
3
4
3
3
4
3
3
4
4
4
3
3
2
2
3
2
2
2
3
1
2
2
3
2
3
2
2
3
2
3
2
3
3
3
4
3
3
2
2
2
3
3
3
3
3
2
3
3
3
3
2
3
2
3
3
3
2
4
4
3
2
3
3
2
3
3
1
4
3
4
2
3
3
2
3
3
3
3
3
4
3
2
3
3
4
3
3
4
2
2
2
3
3
3
2
3
3
1
1
3
1
1
3
2
2
2
3
2
2
4
2
3
4
2
4
3
3
3
3
3
1
3
3
3
3
3
3
3
2
2
3
3
2
2
3
2
2
2
2
3
2
2
1
1
3
2
3
3
3
3
2
2
2
3
3
3
2
4
1
4
3
2
3
2
1
1
2
3
3
2
2
2
Happy Widesti
Victoria Evangelista
Bram Sivadjati
Kartika Dianing R.
Maria Intan Permata
Dara
Anestasia Yuliani
Santa Maya Pramusita
Nanik Priyanti
Oei Steffi
Natalia
Rizza Faoziah
Avinda Kusuma W.
Raissa Junita
Lena
Retrianti Prasetya
11201204
4
11201204
7
11201205
0
11201205
1
11201205
2
11201205
4
11201205
5
11201205
6
11201205
9
11201206
0
11201206
5
11201206
6
11201206
7
11201207
1
11201207
4
11201207
5
3
3
4
2
4
4
3
4
4
3
3
4
4
3
3
2
2
3
2
3
3
3
3
2
3
3
3
3
3
2
3
3
3
3
4
3
3
4
3
3
4
4
4
3
3
3
2
4
2
3
4
3
4
2
3
3
4
4
3
3
1
1
1
1
1
2
1
2
1
1
1
2
1
1
1
3
1
4
3
3
4
2
4
2
2
3
4
3
4
3
4
3
4
2
2
3
3
3
2
4
4
3
3
4
4
2
2
3
2
3
3
3
2
2
3
2
2
3
3
2
3
4
4
4
3
4
2
4
3
3
2
4
1
2
3
3
2
4
2
3
3
2
3
3
3
3
3
3
4
3
3
1
4
2
2
3
2
4
2
4
2
3
3
3
2
3
3
4
3
3
4
3
4
3
3
3
3
3
3
3
3
2
4
2
3
4
3
2
3
3
2
3
3
3
4
2
3
4
3
3
4
2
3
2
4
3
3
3
3
2
4
4
3
3
3
2
3
2
3
3
4
3
2
3
4
3
3
3
2
2
3
2
3
2
3
3
3
2
3
3
Chang Sese
Annisa Putri
Femy Ngailo
Ivana Erlinda
Laurensia Pamela
Fionna
Helena Lorentia
Hannasista Y.
Alfonsius Mario E.
Yulio A. P.
Sulistyowati
Laura Mezba
Erella Elen C.
Jap Victoria
Venda Vista T.
11201207
6
11201207
9
11201208
3
11201208
6
11201208
7
11201209
0
11201209
2
11201209
5
11201209
7
11201209
8
11201210
2
11201210
9
11201211
1
11201211
3
11201211
9
11201212
1
2
2
3
3
3
3
3
3
3
3
3
3
3
2
3
4
4
3
4
3
4
3
4
3
3
3
3
2
3
3
2
2
3
2
3
3
2
3
3
3
3
3
3
3
1
4
3
3
3
3
4
3
3
3
3
2
3
4
3
3
4
3
2
4
4
3
3
4
3
4
4
4
4
3
3
2
2
3
3
3
3
2
3
1
3
2
3
3
3
2
3
1
1
1
3
1
1
3
1
1
1
2
1
1
2
1
2
4
3
3
3
3
3
2
2
3
3
3
3
2
4
1
3
2
3
4
3
4
2
3
3
3
3
3
2
1
3
4
3
3
4
3
3
4
4
4
4
4
4
1
3
1
4
3
3
2
2
2
3
3
3
4
2
3
2
3
1
4
3
3
2
2
2
3
3
3
4
2
3
2
1
1
2
3
2
2
3
2
2
4
3
4
4
4
1
1
1
3
2
3
4
2
3
2
3
2
2
4
3
3
2
3
4
3
3
3
3
3
2
3
3
3
3
3
3
2
2
3
2
2
3
3
3
2
2
3
3
3
3
2
Pawinda Tiffany P.
K. Handayani
Oviera Maulia G.
Yanti Ratna P.
Frenky Batunan
Hapsari C. Hanandya
Kurnia
Yayas K.S.
Azaria
Dian Nielson P.
Ninda Putri
Hilda Karunia S.P.
Luky Rias Pambudi
Naomi Tade Djo
NoName
11201212
6
11201212
9
11201213
0
11201213
1
11201213
2
11201213
3
11201213
5
11201213
6
11201213
7
11201213
9
11201214
1
11201214
3
11201214
6
11201214
7
-
2
2
4
3
3
4
3
3
3
3
3
3
2
3
2
3
1
3
2
2
2
2
3
2
1
1
2
2
4
2
3
1
3
1
2
4
2
3
2
3
3
3
3
3
2
3
2
2
3
3
3
2
3
2
2
2
2
3
2
2
2
1
3
2
3
4
3
4
2
3
4
4
3
3
2
4
3
4
4
3
4
3
3
3
3
3
3
3
3
3
3
2
2
3
2
2
2
3
2
1
1
2
2
4
2
2
1
2
3
3
3
2
3
2
2
3
3
3
3
2
1
2
3
3
3
3
3
3
3
3
3
3
3
3
2
3
3
4
3
4
3
3
4
3
3
3
4
3
3
4
2
1
1
3
3
3
2
3
2
3
3
3
2
3
3
1
2
4
3
3
4
2
3
2
3
4
4
4
3
1
3
1
3
1
2
4
2
3
2
3
3
3
3
3
2
1
2
2.64
9
1
1
2.06
8
2
2
3.16
2
3
2
2.58
1
2
2
2.79
7
2
3
3.17
6
3
3
2.54
1
2
2
2.91
9
2
2
2.25
7
4
2
2.85
1
3
3
2.86
5
4
3
3.08
1
4
2
2.94
6
4
2
2.87
8
1
2
2.39
2
Appendix C
Opinions about the benefial and harmful effects of multiple-choice questions
1. Beneficial: I don't need to study hard because I just need to read the material.
2. I think MC is easy to do, but it doesn't good way to measure students' understanding.
3. Beneficial: Practice, save time. Harmful: Reduce to give our own answer.
4. Make me more easier to answer questions.
5. 6. Multiple-choice questions decrease the anxiety. If I can't answer the question, I can pick one of the answer randomly.
7. We can save a lot of time to check our answer again and we can increase our ability with new vocabulary.
8. That can help me to remember the answer and elimination wrong answers. It has bad effect because easy to cheat.
9. Multiple-choice is needed to save the time in doing the test because sometimes we don't have enough time to do many questions.
Multiple-choice can also decrease our ambiguity. We can compare the answer from A until E and decide which one will be the best.
But, sometimes multiple-choice can confuse us too if the option almost same.
10. The multiple-choice most have some key word of the answered from the question. And make it more easier to understand.
11. If we do multiple-choice test sometimes we confused about the answer because there are many choices that may be we think
that it was the best answer.
12. For the MC question, it's hard to choose if there is misstyping on its answer.
13. 14. If don't have any idea about the answer, we can choose one of the choices.
15. Multiple-choice questions make student easier to cheat.
16. 17. Sometimes, in some subject, multiple-choice is the best way to answer the question. Multip
Multiple-choice Questions
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Merry Tan
112005156
ENGLISH DEPARTMENT
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2013
The Perceptions of English Department Students on the Use of
Multiple-choice Questions
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Merry Tan
112005156
ENGLISH DEPARTMENT
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2013
i
COPYRIGHT STATEMENT
This thesis contains no such material as has been submitted for examination in any
course or accepted for the fulfillment of any degree or diploma in any university. To
the best of my knowledge and my belief, this contains no material previously
published or written by any other person except where due reference is made in the
text.
Copyright@ 2013 Merry Tan and Dra. Martha Nandari S. Handoko, MA.
All rights reserved. No part of this thesis may be reproduced by any means without
the permission of at least one of the copyright owners or the English Department,
Faculty of Language and Literature, Satya Wacana University, Salatiga.
Merry Tan:
iii
Introduction
There are pros and cons perceptions in the society about multiple-choice
test questions (MCQs). Many people think that MCQs are not quite effective for the
educational world because it is hard to measure students‟ ability level. However,
MCQs are the most popular question type used in teaching and learning process
(Oosterhof, 2001, Tozoglu et al., 2004). According to Lee (2011) as cited in Ihsan
Ibadurrahman, MCQs are the most commonly used testing type in many academic
achievement tests but MCQs are also the most criticized type. Some students might
be more able to perform on MCQs better than other question types. Generally,
students' perceptions or attitudes toward the types of question could differ according
to their outlooks about their performance on the test. Moreover, although most of the
students report that multiple-choice questions are easier than the other question
types; they could consider multiple-choice questions are more difficult than the other
question types in some situations (Holtzman, 2008). The explanation of those results
might connect with students‟ belief about the process of preparing for this type of
questions. So, they perhaps think that with objective questions, it is easier to study
the material. Zeidner (1987) stated that students' perceptions' toward test types can
differ in terms of different works, such as how they perceived interestingness,
usefulness, trickiness, fairness and other elements.
Multiple-choice questions also have advantages and disadvantages in
teaching and learning process. It is supported by the statement from Ben-Simon,
Budescu and Nevo, (1997) that stated “Multiple-choice tests also have advantages
which are mentioned that they easy to score, can be described objective in terms of
their scoring, easy to computerize specially in term of scoring process, and more
importantly it is easy to cover the objectives of the course through this type of
1
exams.”. Overall, there are many problems that appeared from this type of questions,
such as the guessing issue, the difficulty to measure the real students‟ ability, and the
process of preparing questions which is not easy because it needs time and efforts
(Oosterhof, 2001). Some researchers still argued that this type of questions can
measure the complexity of academic processes effectively (Mehrens and lehman,
1984). The literature indicates that students prefer this type of exam comparing with
other types exam (Zeidner, 1987; Choi, 1998; Birenbaum and Feldman, 1998). The
details of these results might linked with students‟ perceptions about the process of
preparing for this type of exam, as the result, they perhaps believe that with objective
exams it is easier to study the material, and they might think that multiple-choice
type question is more fair than the other types question.
Zeidner (1987) tried to study secondary school students' perceptions of
essay and multiple choice type questions, the participants of his study consisted of
174 male and female students. Two instruments were applied in this study. First, he
represented students' perceptions in terms of ten scopes: difficulty, complexity,
clarity, interest, trickiness, fairness, value, success expectancy, degree of anxiety
evoked, and feeling at ease. The second instrument was used as standard instrument,
to observe the validity of the major instrument, this instrument included seven
scopes. In the second tool the students were asked to answer whether they prefer
essay or multiple choice type questions. The results of this study revealed that
secondary school students preferred multiple-choice question more than essay
question.
Furthermore, Nassar (2006) investigated the Hashemite University students‟
training on skills related to their achievement on true-false, multiple choice,
completion, and essay questions. He found that the training was positive in
2
developing students' skills in dealing with the mentioned questions. In general,
Nassar observed that the biggest problem that students dealt with was obtained in
students' achievement on essay questions. He explained that this result due to
students' perceptions of essay questions before and after the training. As the result,
students might have negative perceives toward this type of questions because they do
not have sufficient qualifications of dealing with such questions.
The study of English Department (ED) students‟ perceptions about the use
of multiple-choice type questions on the learning process is being focused to
determine the beneficial and harmful effects on doing the MCQs. This study surely
has a major effect to establish and to recognize the important of multiple-choice
questions. The project‟s goal is designed to help students improve their achievement
level, to immobilize the misunderstanding toward multiple-choice questions, and to
decide the validity of multiple-choice test questions for sure. This goal can be
achieved through the teacher who can teach the students the tricks on doing the
multiple-choice test questions and English Department/ED students‟ help to talk
about their perceptions toward the use of multiple-choice test questions on the
learning process. The output of this study is various perceptions about the use of
multiple-choice test questions on the learning process that can make the students
think about the benefits and harmful of multiple-choice test questions and
disseminate by renewal the negative perceptions to positive perceptions.
The research questions to be answered are:
1. How do the English Department (ED) students think about the use of
multiple-choice questions in their tests?
3
2. According to the English Department (ED) students, what are the
beneficial effects and the harmful effects of multiple-choice questions to
their learning process?
Multiple-choice questions are basically recognition tasks because students
must identify the correct response. The multiple-choice questions are supposed to be
a very flexible part of assessment format that are used to measure knowledge, skills,
abilities, values, thinking skills, etc (Haladyna, 1999; McDougall, 1997). Besides
that, a test usually consists of a number of questions and the students must choose
the right answer among the number of choices. Therefore, students are trained to use
their brain to think logically. Then, multiple Choice Questions (MCQs), also called
as an objective test, is one of the test types in language assessment where
participants are typically asked to respond by giving one correct answer from the
available options (Hughes, 2003). Hughes (2003, pp.76-78) stated that multiplechoice questions are all receptive or selective, response items chosen by the testtaker from a set of responses or usually called as a supply type of response rather
than creating a response. Multiple-choice questions have a stem, which includes a
stimulus, and options usually between three and five to choose. One of the options is
the correct response, while the others are the distractors.
As we know, there are many beneficial effects of the use of multiple-choice
questions on teaching and learning process. According to Haladyna (1999), multiplechoice questions are considered to be beneficially an efficient and effective way to
assess a wide range of knowledge, skills, attitudes and abilities. Multiple-choice tests
can also be a useful tool to develop students‟ thinking skill (Haladyna, 1999).
According to Hughes (2003, pp. 76-78), multiple-choice questions are
practical and reliable. With their programmed correct responses and time-saving
4
scoring procedures, it is easier for the teacher on grading and scoring. It is also easier
for the students to answer the MCQs because in each question, there is a correct
answer in between the options for sure.
Generally people have the opinion that MCQs only deal with lower-order
analytical skills that require knowledge, application and comprehension only, but not
the high-order skills including critical thinking and problem solving skills.
According to William (1996) & Mehrens (1998), the latest test appliances spending
multiple-choice questions to test higher order analytical skills or which include
analysis, synthesis, and evaluation to display students‟ point. Higher order analytical
skills make the limited errors in doing multiple-choice questions. To develop these
skills, students can debate, write and master the structured argument. Smothering
education is needed but of course with high responsibility. The example of this
analytical skill is students have to collect the data by using the internet. McGuinness
(1999) points out that higher order analytical skill includes categorizing and
analyzing information, drawing conclusions from the information, „brainstorming‟
new ideas, and solving the problem. Besides that, he also added that higher order
analytical skill includes determining cause and effect, evaluating options, planning
and setting goals, monitoring progress, making decision, and reflecting on one‟s own
progress.
William (1996) & Mehrens (1998) mentioned further that people have
discarded the statements that multiple-choice format is competent of dealing with
lower order skills of recall or recognition. Lower order skills are used if the learner
does not validate, question, or evaluate, and in the end the information obtained. The
example of this skill is when students have to collect the information by using
5
electronic textbook. According to McGuinness (1999) collecting information is the
lower order analytical skills.
Although some people think that multiple-choice questions are effective for
the teaching and learning process, actually there are also many harmful effects that
students and teacher faced from the use of multiple-choice tests. Some people claim
that multiple-choice questions are too easy for the students. It makes the students
lazy to study. Multiple-choice questions are easiest to write when there is a
definitively right or wrong answer (Bloom, Benjamin B, 1956).
The bad effect of multiple-choice questions that clearly appear on the
learning process is the guessing issue. The score is unpredictable because this is just
depending on the luck. However, the students would usually prefer multiple-choice
questions than the essay or other question types because of that reason. Students
think that MCQs are helpful when they forgot the material because they can guess
the answer. According to Espinosa & Gardeazabal (2009), students are motivated to
have a high possibility to guess in doing multiple-choice questions. Besides that,
candidates may be able to pick the correct answer without having to read the passage
or understanding the meaning in it (Lee, 2011: 31). Similarly, a correct answer can
also be the result of eliminating incorrect answers, without the candidates knowing
the right answer in the first place (Woodford & Bancroft, 2005).
People think that multiple-choice questions are always objective and highly
reliable for most students to improve the skills, but this makes the students idle in the
learning process. They just have to choose one of the options that they think is the
true one whereas we don‟t know if they understand the lesson or not. Related to the
learning skills‟ disability, people argued that students can improve their marks on
multiple choice tests but practice in taking multiple-choice questions and test-taking
6
strategies will not give learners with the most effective ways of improving their
language ability (Hughes, 2003).
Multiple-choice questions are time consuming. Just like Burton, Sudweeks,
Merrill, & Wood (1991) argued that good multiple-choice questions are more
difficult and more wasting the time to make than other types of test questions that
require skills. This skill, however, may be increased through study, practice, and
experience.
Multiple-choice questions are known as a simplest kind of question types to
create, but it is extremely difficult to design appropriately. Hughes (2003, pp. 76-78)
mentioned that the weaknesses of multiple-choice questions are only technique tests
that recognize knowledge and severely limit what can be tested. It mentioned that
guessing and cheating may have a significant effect on test scores.
Adapted from Gronlund (1998, pp. 60-75) and J.D. Brown (1996, pp. 5457), there are four guidelines for designing multiple-choice questions for both
classroom-based and large-scale situations. The first one is designing each item to
measure a specific objective (the comprehension of wh-questions. Next is stating
both stem and options as simply and directly as possible. To make it as a good
multiple-choice question, it‟s better to make it not too wordy and get directly to the
point. The third one is being sure that the intended answer is clearly the only correct
one. Then, the last guideline for designing multiple-choice items is using item
indices to accept, discard, or revise items.
Eliminating not premeditated possible answers is often the most difficult
problem of handling multiple-choice items. According to Gronlund (1998, pp. 6075) and J.D. Brown (1996, pp. 54-57), There are three indices that appeared on
measuring items to choose the proper selection and suitable multiple-choice items on
7
a test. First is item facility (IF). Item facility is the scope to which an item is easy or
difficult for the planned group of test-takers. The teacher should classify the
questions carefully to make it not too difficult and too easy because it‟s invalid to
separate the high- and low-ability test-takers. Second is item discrimination (ID).
Item discrimination is the scope to which an item differentiates between high- and
low-ability test takers. Poor ID would appear when the high- and low-ability students
score equally well because it did not discriminate between the two groups. The last
index is distractor efficiency which is related to item discrimination. The example is
in a test at least two distractors attracting some attention from lower ability test
takers.
The Study
In order to answer the questions of this study, the quantitative method,
namely survey research, was used. Then, to convince and analyze my own
perspective of my study, I used qualitative descriptive method with an open-ended
style. Open-ended (or “open question”) is a question where the researcher doesn‟t
provide the respondent with a set answer from which to choose. Rather, the
respondent is asked to answer "in their own words". In this case, I used an essay
question to provide students‟ perspective. The participants of this study were
seventy-four students from basic grammar classes in the English Department of
Satya Wacana Christian University in the second semester of the academic year
2012-2013.
Data Collection Instrument
I developed a questionnaire as the instrument to collect the data. This
instrument consisted of two sections. The first section consisted of the fifteen
8
questions which the students had to choose for the best answer, and the second
section consisted of an essay question which asked about students‟ perceptions of the
use of MCQs in their tests. The questions of perceptions‟ were prepared by the
researcher and the ideas of these questions were taken from some researchers‟
perceptions toward the use of MCQs. The first three questions of the first section
were made to find out the students‟ perceptions on the difficulty and possibility of
guessing, cheating, and ambiguity of MCQs. The same procedure was developed
with the questions which were constructed to measure the students' skills by using
MCQs on their tests. Next parts of the questions were developed to find out students‟
perceptions toward the benefits of the use of MCQs. The instrument was constructed
as a Likert-type rating scale on a four-point range that was scored and categorized as
following: Strongly Agree=4, Fairly Agree=3, Fairly Disagree=2, Strongly
Disagree=1. Higher scores on the questions indicate more favorable dispositions
toward the use of multiple-choice questions type under consideration. Finally, the
writer counted the results one by one each questions.
Data Collection Procedures
In the beginning, the students from two Basic Grammar classes (thirtyseven students of each class) were selected randomly. The number of all of the
students who were in those classes was seventy-four. The questionnaire was
administered to all of the students selected for this study. The students from one of
the selected Basic Grammar class were participated all because no absent student at
that time, however one of the Basic Grammar class which participated in this study
was selected randomly from the rest three classes of Basic Grammar class students.
They were informed about the list of beneficial and harmful effects of multiplechoice questions before responding to the questionnaire. Students were expected to
9
choose whether they agreed or disagreed with the list of beneficial and harmful
effects of MCQs. Students respond to the instrument with the name and student ID
written on, and they need in average 5-10 minutes to accomplish the task. The
process of collecting the data most of the time was done by the researcher or with the
help of friends or some teachers who were teaching in the selected classes.
Quality of Data
For the purpose of examining the validity of the instrument (face validity
evidences), I used SPSS to make sure for the validity. After inputted the data to
SPSS, I used the descriptive reliable to measure it. After got the reliability statistics
and item-total statistics, the questions could be determined the validity and
reliability. If the value of Cronbach‟s Alpha was > 0.70, it means that the test was
reliable. And if the value of corrected item-total correlation was > r table, it means
that the test was valid.
Data Analysis Procedure
Besides that, an essay question was made to find out if there were other
opinions about the use of MCQs to support the answers in the first section (list of
questions). Sample questions from the instrument are as follow: "It is easier to get
high score in MCQs", "MCQs are practical and reliable", "MCQs decrease the
anxiety", "and MCQs differentiate good, average, and weak students". In order to
answer the research questions, I explained questions with the dominant answers one
by one with the supporting statements from the essay question.
Discussion
In order to describe the English Department students‟ perceptions toward
the use of multiple-choice questions, there were fifteen questions given to them.
10
Those questions were about the advantages and disadvantages of multiple-choice
questions.
Table 1. MCQs increase the chance of guessing.
SA
FA
FD
13
32
SA = Strongly Agree
19
SD
10
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As shown in the first question, there were thirty-two participants who fairly
agreed that multiple-choice questions increased the chance of guessing. Some of
them thought that the chance of guessing was the good idea to raise their mark and to
help when they forgot the material. Aprilia L, one of the students in the class of basic
grammar, said that “If we don’t have any idea about the answer, we can choose one
of the choices”. Not much different with Aprilia L, Laurensia Pamela assumed that
“If we really don’t know the answer, we can guess the answer and if we are lucky we
ca n get the great score”. That means those students perceived the positive points of
guessing. Multiple-choice assessments with a high number of stems and options
helped the students to get at least 25% if the test-takers picked one of the options
continuously by purely guessing. Intan Ariani argued that “Multiple-choice
questions give a chance for the students to guess if they really don’t know the
answer. It is really helpful, but on the other hand, students become lazy to study
because they still have at least 25% chance to guess the right answer.”
Table 2. MCQs reduce the chance of cheating.
SA
FA
FD
5
18
SA = Strongly Agree
28
SD
23
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
When asked whether multiple-choice questions reduce the chance of
cheating, it was revealed that the participants in this research fairly agreed but not a
11
few who disagreed with the question. Some students perceived that “Multiple-choice
questions will increase the chance of cheating.” It had been spreading over the world
that multiple-choice questions could not be separated with the issue of cheating.
Yayas K.S. also said that “multiple-choice will increase the chance of cheating with
other students.”
Table 3. MCQs have more options that help the students in comparison and increase
the ambiguity.
SA
FA
FD
SD
29
32
SA = Strongly Agree
9
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Towards question number three “multiple-choice questions have more
options that help students in comparison and increase the ambiguity” students
strongly agreed. Many of them also argued that “multiple-choice questions are really
confusing and so ambiguous because the answer is quite similar and it’s trapping.”
Actually, the confusing part was located in the question construction. Focusing on
important content was not enough.
Table 4. MCQs improve students‟ language skills.
SA
FA
FD
6
37
SA = Strongly Agree
25
SD
6
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As predicted, the students fairly agreed with the fourth question which
asked if multiple-choice can improve students‟ language skills. One of the students
argued that “we can increase our ability with new vocabulary.” Although it was
difficult to increase the students‟ language ability with multiple-choice questions,
actually it was simpler when the students measured their language ability.
Table 5. MCQs develop students‟ thinking skills.
12
SA
FA
7
47
SA = Strongly Agree
FD
18
SD
2
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Then, as shown in the fifth question, the participants showed their interest
and agreement in this question (multiple-choice questions develop students‟ thinking
skills). Kartika Dianing R wrote that “Multiple-choice questions can develop
students’ thinking skills.” Similar to Kartika, Raissa Junita also said that “Multiplechoice questions can help her thinking skills.” There were many positive opinions
about developing students‟ thinking skills.
Table 6. MCQs make students save time in doing the test.
SA
FA
FD
SD
27
34
SA = Strongly Agree
12
1
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Concerning question number six about time-saving in doing the test, the
students strongly agreed with that idea. Nadia Angela told that “We can save a lot of
time to check our answer again.” Not only Nadia who argued that multiple-choice
questions make students save time in doing the test, but also Santa Maya Pramusita
said that “The benefit of multiple-choice questions is we don’t have to write too
much, so it can save the time.”
Table 7. MCQs encourage the students to study.
SA
FA
FD
3
37
SA = Strongly Agree
31
SD
3
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Next focusing on question number seven about multiple-choice questions
that can encourage the students to study, again the students fairly agreed. Chang Sese
assumed that “Multiple-choice sounds like a riddle that makes me curious with the
13
answer, so it motivates me to study harder. I just need to take notes and study the
important points for my strategies.” However, there were some statements from the
participants that said “Multiple-choice makes students lazy to study.”
Table 8. MCQs are easier to get the high score.
SA
FA
FD
15
39
SA = Strongly Agree
19
SD
1
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
About the question number eight, the participants fairly agreed that
multiple-choice questions enabled them to get the high score easier. Rizza Faosiah
said that “Multiple-choice questions help students to easier get the high point.”
Same with Rizza, Laurensia Pamela also said that “If we are lucky we can get the
great score.” In fact, multiple-choice questions were easier to get score because
there were more options that made students easier to do the test.
Table 9. MCQs measure the real students‟ ability.
SA
FA
FD
2
24
SA = Strongly Agree
39
SD
9
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Dealing with question number nine about measuring the real students‟
ability, the participants also fairly agreed. Naomi Tade Djo agreed that “Multiplechoice questions can measure her real ability because she can learn about problem-
solving.” The ability to find out and to solve the stressed problem was more
important.
Table 10. MCQs are practical and reliable.
SA
FA
FD
10
SA = Strongly Agree
47
13
SD
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
14
As shown in the next question that stated that multiple-choice questions are
practical and reliable, the participants agreed. Anissa Putri argued that “Multiplechoice is a simpler way to answer the questions.” Another similar opinion came
from Jap Victoria, who said that “MCQs are quite helpful for students in the test
because it is easier compared to essay.”
Table 11. MCQs are valid.
SA
FA
12
44
SA = Strongly Agree
FD
14
SD
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
The next statement was about multiple-choice questions that were valid.
The participants fairly agreed with. Most of the students answered that “Multiplechoice questions are valid.”
Table 12. MCQs are objective.
SA
FA
17
46
SA = Strongly Agree
FD
11
SD
0
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
As expected, in question number twelve, “Multiple-choice questions called
as an objective”, it was revealed that the participants in this research totally agreed.
The students agreed that “Multiple-choice questions are helpful for the students. It’s
also easier to do than the other type of questions.” It could not be denied that
multiple-choice questions were so helpful especially for the students and the teacher
too.
Table 13. MCQs are easy to do.
SA
FA
FD
15
SD
18
38
SA = Strongly Agree
14
4
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
There were a lot of the participants who agreed that multiple-choice were
easy to do. It was just like what Jessica said, “In my opinion multiple-choice makes it
easier for the students to answer .” Almost similar with what Jessica said, Dara also
argued that “It’s easier to do the exercise with multiple-choice because we just can
learn the important point, but for essay questions we have to think harder.” It means
that multiple-choice questions type is easier than the other question types.
Table 14. MCQs decrease the anxiety.
SA
FA
FD
9
50
SA = Strongly Agree
12
SD
3
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
The result of question number fourteen which asked if multiple-choice
questions decrease the anxiety was agreed. La Valeroe Novianne agreed that “MCQs
decrease the anxiety.” Similar with the students, Snow (1993) said that “MCQs
reduce students’ anxiety.”
Table 15. MCQs differentiate good, average, and weak students.
SA
FA
FD
SD
6
26
SA = Strongly Agree
33
9
FA = Fairly Agree FD = Fairly Agree SD = Strongly Disagree
Last question was the very crucial question. The participants were asked
about MCQs that could differentiate good, average, and week students, and they
fairly agreed. Luky Rias Pambudi argued that “The discrimination of good, average,
and week students happened sometimes, because sometimes also good, average, and
week students get the similar score that make it harder for the teacher to
differentiate the ability.”
16
As expected, almost all of the students preferred multiple-choice questions,
even for the higher-achiever students. It seems that students have built their thought
that multiple-choice is easier to do because it has more options which are directed to
the real answer. Personally, I agree with my participants‟ perceptions and I think that
my participants weren‟t only copy paste from my statements only, because they
made their own statement and there was a statement that made me realize with the
fact that “always pick answer C and you‟ll be right about 25% of the time”. As we
know in general, it is impossible because the teacher should make the questions more
complex and unpredictable in scoring. However, the researches convince that it is
possible and even it is a must that at least 25% of the score will be given if the
students choose “C” all of the time. It is very interesting because it will happen if the
teachers are competent in the process of questions‟ making.
Conclusion and Suggestions
My study surely was made to establish and to recognize the important of
multiple-choice questions. In this study, students‟ perceptions on the use of multiplechoice questions were generally positive. Out of the four-point range, the students
chose strongly agree and fairly agree, instead of strongly disagree and fairly disagree
each question. As founded in this study, the students preferred multiple-choice
question type than the other question types such as essay question or fill in the blank
question type. Shockingly, there are some contradictory findings in this study.
Almost a half of the participants agreed that MCQs reduce the chance of guessing
and cheating. Like what we discussed before, half of them agreed that MCQs
increase the chance of guessing and cheating, but more of them chose to disagree
with it. Some revealed that there were no relation between MCQs and the chance of
17
guessing and cheating. Some stated that although there were options on the question,
if they chose the wrong answer, the point was obviously zero. As a conclusion, it
doesn‟t mean that if the students choose to guess and cheat in the test, they will get at
least 25% of the score no matter of what. The previous result might indicate that the
students perceive multiple-choice as a positive and the beneficial one. Although
there were pros and cons about multiple-choice around the participants, most of them
agreed that multiple-choice is very helpful. It is very obvious either through the
results of this study or the results of the other studies in literature that students in
general have positive thoughts toward multiple-choice questions. Surprisingly,
students fairly agreed with the statement “MCQs measure the real students‟ ability”
but heed that MCQs increase the chance of guessing. It doesn‟t mean that students
indicate guessing as a part of real students‟ ability, but sometimes not only real
students‟ ability that needed in MCQs but also the sense of proper guessing should
be needed. It proves that even the high-achievers also try to guess sometimes. The
more interesting result was that although students preferred multiple-choice
questions more than the other question types. They thought that multiple-choice
questions were the most confusing question type.
Implication
To make sure that students succeed in their multiple-choice questions exam,
the teachers should motivate the students to study harder when they have exam with
multiple-choice questions. It can improve students‟ thinking skills and language
skills with deep learning.
Suggestions
Due to the results of the current study and to mentioned results in the
literature, the following recommendations could be presented:
18
1. The students cannot underestimate the multiple-choice questions
because the more options means the more confusing they can be.
2. The teachers should make a report about students‟ perceptions of
multiple-choice questions. Since students' scores on this question type
is used to take decision about their real ability.
3. The teachers must be careful in making the good multiple-choice
questions. So that they are valid and reduce the ambiguity.
Limitation
The participants of this study were Basic Grammar students' of the first
grade in English Department of Satya Wacana Christian University of the academic
year 2012-2013. Also the results of this study should be interpreted in the context of
its definitions of the variables, its instrument, and the research method used to
answer the research questions.
Further Research
Other researchers could describe the beneficial and harmful effects of other
question types in the next research. They also could compare multiple-choice
question type with the other question types.
Acknowledgement
Foremost, I would like to express my sincere gratitude to my advisor Bu
Martha Nandari for the continuous support of my study and research, for her
patience, motivation, enthusiasm, and enormous knowledge. Her guidance helped
me in all the time of research and writing of this thesis. I could not have imagined
having a better advisor and mentor for my study. I also would like to thank my
second reader, Bu Maria Eko for her advices, suggestions, and tolerance for the ease
of my work.
19
Besides my advisor, I would like to show my biggest gratitude to best of
the best advisor of my life, God, You help me pass all of the difficulties I faced.
Please bless me for the next steps of my life too. Next, I would like to thank my
family: my lovely parents for the praying and the words that motivate me to pass this
study. Thanks to my cousins who help me a lot: Didi, Aweng, Chang Sese, Jessica,
Silvia, Agnes, Jenni, Weni, Hia Rein, Mengkhoi hia, Sok Pheng hia, LingCe, Cehu,
EngCe, Aso, etc. For my aunts and uncles who support me. I love you all.
I would like to thank my best friends: Alce and Izni for sharing the
difficulty and laugh together. Thank to my forever friends: TuaCe, TuaHia, Apang,
Bos, Hendi, Angie, Johny, Oni, and Wendi for filling my beautiful life. I always love
you. Wish you all the best and joy. Last, for my boarding house‟s friends, Maria,
Ani, Cepy, Mbak Ita, Pinot, Desi, Uning, Rosi, Marlyan, Wicha, Hery, Susy, Ita,
Jessica, etc, thank for everything. I will never forget you girls. Thanks for the
support, the joy, and the help when I needed something. Go… Go… Dipo 74A.
20
References:
Aiken, L. R. (2000). Psychological Testing and Assessment (10thEdition). Boston,
MA: Allyn and Bacon.
Bachman, L. & Palmer, A. (1996). Designing and developing useful language tests.
Oxford: Oxford University Press.
Ben-Simon, A., Budesco, D.V. and Nevo, B. 1997. A comparative study of measures
of partial knowledge in multiple-choice tests. Applied Psychological
Measurement, 21 (1): 65-88.
Biggs, J. B. (1973). Study behavior performance in objective and essay formats.
Australian Journal of Education, 17, 157-167.
Brown, J.D. (1996). Testing in language programs. Upper Saddle River, NJ:
Prentice Hall Regents.
Bush, M. (2001). A multiple-choice test that rewards partial knowledge. Journal of
Further and Higher Education, 25 (2), 157-163.
Cannon, R.A. and Newble, D. (1983) A Handbook for Clinical Teachers, Lancaster,
MTP Boston: p 97-105.
Carey, J. (1997). Everyone knows that E = MC2, who can explain it? Business Week,
3547, 66-68.
Chatterji, M. (2003). Designing and Using Tools for Educational Assessment.
Boston, MA: Allyn and Bacon.
Dufresene, R. J., Leonard, W.J., & Gerace, W.J. (2002). Making sense of students‟
answers to multiple-choice questions. The Physics Teacher, 40, 174-180.
Gronlund, N. E. (1998). Assessment of Student Achievement. Sixth Edition. Boston,
MA: Allyn and Bacon.
Holder, W. W., & Mills, C. N. (2001). Pencils down, computer up: The new CPA
exam. Journal of Accountancy, 191 (3), 57-60.
Hughes, A. (2003). Testing for language teachers (2nd edition). Cambridge:
Cambridge University Press.
Johnson, D. W. & Johnson, R. T. (2002). Meaningful Assessment: A Manageable
and Cooperative Process. Boston, MA: Allyn and Bacon.
Kreig, R.G., and Uyar, B. (2001). Student performance in business and economics
statistics: Does exam structure matter? Journal of Economics and Finance, 25
(2), 229-241.
Lee, J. (2011). Second Language Reading Topic Familiarity and Test Score: TestTaking Strategies for Multiple-Choice Comprehension Questions (PhD.
Thesis). Retrieved from ProQuest Dissertations and Theses. (Accession Order
No. 3494064).
Marshall, J. C., & Hales, L. W. (1971). Classroom test construction. Reading MA:
Addison-Wesley
http://www.utexas.edu/academic/mec/research/pdf/writingmcexamshandout.pd
f
21
McKeachie, W. J. (1999). Teaching Tips: Strategies, Research, and Theory for
College and University Teachers (10thEdition). Boston, MA: Houghton Mifflin
Company.
McTighe, J. & Wiggins, G. (2005). Understanding by design (2nd ed). Alexandria,
VA: Association for Supervision and Curriculum Development.
Popham, W. J. (2000). Modern Educational Measurement: Practical Guidelines for
Educational Leaders (3rdEdition). Boston, MA: Allyn and Bacon.
Stoynoff, S. & Chapelle, C. (2005). ESOL tests and testing. Alexandria, VA:
TESOL.
Snow, R. E. (1993). Construct validity and constructed-response tests. In E. B.
Randy, & C. W. William (Eds.), Construction versus choice in cognitive
measurement. Hillsdale, NJ: Lawrence Erlbaum, 45-60.
Trice, A. D. (2000). A Handbook of Classroom Assessment. New York: Addison
Wesley Longman, Inc.
Zeidner, M. (1987). Essay versus multiple-choice type classroom exams: The
student‟s perspective. Journal of Education Research, 80 (6), (July/August),
352-358.
22
Appendix A
Questionnaire:
I am doing a research about the students’ perceptions on the use of multiple-choice questions on the learning process. I would like to
know your opinions about the beneficial and harmful effects of multiple-choice questions.
INSTRUCTIONS: Please rate how strongly you agree or disagree with each of the following statements by placing a circle in the
appropriate box. Choose “strongly agree”, “fairly agree”, “fairly disagree”, or “strongly disagree” based on your opinion.
1. Multiple-choice questions reduce the chance of guessing. Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
2. Multiple-choice questions reduce the chance of cheating.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
3. Multiple-choice questions have more options that help
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
the students in comparison and reduce the ambiguity.
4. Multiple-choice questions improve students‟ language
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
skills.
5. Multiple-choice questions develop students‟ thinking
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
skills.
6. Multiple-choice questions make students save time in
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
doing the test.
7. Multiple-choice questions encourage the students to
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
study.
8. Multiple-choice questions are easier to get the high score. Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
9. Multiple-choice questions measure the real students‟
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
ability.
10. Multiple-choice questions are practical and reliable.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
11. Multiple-choice questions are valid.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
12. Multiple-choice questions are objective.
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
Strongly Agree
Fairly Agree
Fairly Disagree
Strongly Disagree
13. Multiple-choice questions are easy to do.
14. Multiple-choice questions decrease the anxiety.
15. Multiple-choice questions differentiate good, average,
and weak students.
What do you think about the beneficial and harmful effects of multiple-choice questions on your learning process? Please give your
opinion about it and specify if you have the other ideas about characteristic of multiple-choice questions.
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………………………………………………………
…………………………………………………………………………………………………………
Appendix B
Strongly disagree = 1
Fairly disagree = 2
Fairly agree = 3
Strongly agree = 4
Name
Andika W.
Handini S.
Wiwid Wahyuni
Ivana Christina H.
Venny C.R.
La Valeroe Novianne, Lie
Nadia Angela
Sabati Sandy Simonne
Sola Gratia Deo Wiyana
Marcellina A.G.H
Permata Dipaning Ratri
NIM
11200814
6
11201012
5
11201200
3
11201200
5
11201200
6
11201200
8
11201200
9
11201201
1
11201201
2
11201201
3
11201201
6
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q13
Q14
Q15
3
2
1
2
2
3
1
2
2
2
3
3
3
3
2
3
1
3
2
3
4
4
4
1
3
4
3
3
3
1
3
2
3
3
4
4
4
2
2
3
3
3
3
3
1
3
2
3
2
3
3
3
3
3
3
3
3
3
3
2
3
3
4
3
3
3
3
3
3
4
4
3
3
4
3
2
2
3
2
3
3
3
2
2
3
2
3
2
3
2
3
3
3
4
4
4
3
3
3
3
3
3
4
3
3
3
2
4
1
2
4
2
2
1
4
4
4
3
3
2
4
1
4
2
2
4
2
4
1
3
2
3
4
3
2
3
2
4
3
3
3
3
4
3
3
4
4
4
3
2
2
2
3
3
3
2
2
2
2
2
3
2
2
2
3
Joannita Ratih
Wiharmastu
Winda Adelia
Aprilia I
Vania Hendra Gunawan
Lidya
Y. Yubilate. R.
Belinda Gita Puspita
Nova Suciyanti
Jessica
Firsta P.S.
Intan Ariani
Windi Ismitasari
Grace Kartika R.
Sandra Priskila
Yoas Betty K.
Fajarini Prihastuti
11201201
8
11201201
9
11201202
1
11201202
4
11201202
5
11201202
6
11201202
7
11201202
8
11201202
9
11201203
0
11201203
1
11201203
2
11201203
4
11201203
6
11201203
7
11201204
1
2
1
4
4
4
4
2
2
1
2
3
4
3
1
4
3
3
3
3
3
3
3
3
2
3
4
3
4
3
4
3
2
3
3
3
4
3
3
2
3
3
3
3
2
2
4
4
3
2
3
3
2
2
3
3
3
3
3
2
3
2
2
3
3
2
2
3
2
3
4
3
3
4
3
3
4
4
4
3
3
2
2
3
2
2
2
3
1
2
2
3
2
3
2
2
3
2
3
2
3
3
3
4
3
3
2
2
2
3
3
3
3
3
2
3
3
3
3
2
3
2
3
3
3
2
4
4
3
2
3
3
2
3
3
1
4
3
4
2
3
3
2
3
3
3
3
3
4
3
2
3
3
4
3
3
4
2
2
2
3
3
3
2
3
3
1
1
3
1
1
3
2
2
2
3
2
2
4
2
3
4
2
4
3
3
3
3
3
1
3
3
3
3
3
3
3
2
2
3
3
2
2
3
2
2
2
2
3
2
2
1
1
3
2
3
3
3
3
2
2
2
3
3
3
2
4
1
4
3
2
3
2
1
1
2
3
3
2
2
2
Happy Widesti
Victoria Evangelista
Bram Sivadjati
Kartika Dianing R.
Maria Intan Permata
Dara
Anestasia Yuliani
Santa Maya Pramusita
Nanik Priyanti
Oei Steffi
Natalia
Rizza Faoziah
Avinda Kusuma W.
Raissa Junita
Lena
Retrianti Prasetya
11201204
4
11201204
7
11201205
0
11201205
1
11201205
2
11201205
4
11201205
5
11201205
6
11201205
9
11201206
0
11201206
5
11201206
6
11201206
7
11201207
1
11201207
4
11201207
5
3
3
4
2
4
4
3
4
4
3
3
4
4
3
3
2
2
3
2
3
3
3
3
2
3
3
3
3
3
2
3
3
3
3
4
3
3
4
3
3
4
4
4
3
3
3
2
4
2
3
4
3
4
2
3
3
4
4
3
3
1
1
1
1
1
2
1
2
1
1
1
2
1
1
1
3
1
4
3
3
4
2
4
2
2
3
4
3
4
3
4
3
4
2
2
3
3
3
2
4
4
3
3
4
4
2
2
3
2
3
3
3
2
2
3
2
2
3
3
2
3
4
4
4
3
4
2
4
3
3
2
4
1
2
3
3
2
4
2
3
3
2
3
3
3
3
3
3
4
3
3
1
4
2
2
3
2
4
2
4
2
3
3
3
2
3
3
4
3
3
4
3
4
3
3
3
3
3
3
3
3
2
4
2
3
4
3
2
3
3
2
3
3
3
4
2
3
4
3
3
4
2
3
2
4
3
3
3
3
2
4
4
3
3
3
2
3
2
3
3
4
3
2
3
4
3
3
3
2
2
3
2
3
2
3
3
3
2
3
3
Chang Sese
Annisa Putri
Femy Ngailo
Ivana Erlinda
Laurensia Pamela
Fionna
Helena Lorentia
Hannasista Y.
Alfonsius Mario E.
Yulio A. P.
Sulistyowati
Laura Mezba
Erella Elen C.
Jap Victoria
Venda Vista T.
11201207
6
11201207
9
11201208
3
11201208
6
11201208
7
11201209
0
11201209
2
11201209
5
11201209
7
11201209
8
11201210
2
11201210
9
11201211
1
11201211
3
11201211
9
11201212
1
2
2
3
3
3
3
3
3
3
3
3
3
3
2
3
4
4
3
4
3
4
3
4
3
3
3
3
2
3
3
2
2
3
2
3
3
2
3
3
3
3
3
3
3
1
4
3
3
3
3
4
3
3
3
3
2
3
4
3
3
4
3
2
4
4
3
3
4
3
4
4
4
4
3
3
2
2
3
3
3
3
2
3
1
3
2
3
3
3
2
3
1
1
1
3
1
1
3
1
1
1
2
1
1
2
1
2
4
3
3
3
3
3
2
2
3
3
3
3
2
4
1
3
2
3
4
3
4
2
3
3
3
3
3
2
1
3
4
3
3
4
3
3
4
4
4
4
4
4
1
3
1
4
3
3
2
2
2
3
3
3
4
2
3
2
3
1
4
3
3
2
2
2
3
3
3
4
2
3
2
1
1
2
3
2
2
3
2
2
4
3
4
4
4
1
1
1
3
2
3
4
2
3
2
3
2
2
4
3
3
2
3
4
3
3
3
3
3
2
3
3
3
3
3
3
2
2
3
2
2
3
3
3
2
2
3
3
3
3
2
Pawinda Tiffany P.
K. Handayani
Oviera Maulia G.
Yanti Ratna P.
Frenky Batunan
Hapsari C. Hanandya
Kurnia
Yayas K.S.
Azaria
Dian Nielson P.
Ninda Putri
Hilda Karunia S.P.
Luky Rias Pambudi
Naomi Tade Djo
NoName
11201212
6
11201212
9
11201213
0
11201213
1
11201213
2
11201213
3
11201213
5
11201213
6
11201213
7
11201213
9
11201214
1
11201214
3
11201214
6
11201214
7
-
2
2
4
3
3
4
3
3
3
3
3
3
2
3
2
3
1
3
2
2
2
2
3
2
1
1
2
2
4
2
3
1
3
1
2
4
2
3
2
3
3
3
3
3
2
3
2
2
3
3
3
2
3
2
2
2
2
3
2
2
2
1
3
2
3
4
3
4
2
3
4
4
3
3
2
4
3
4
4
3
4
3
3
3
3
3
3
3
3
3
3
2
2
3
2
2
2
3
2
1
1
2
2
4
2
2
1
2
3
3
3
2
3
2
2
3
3
3
3
2
1
2
3
3
3
3
3
3
3
3
3
3
3
3
2
3
3
4
3
4
3
3
4
3
3
3
4
3
3
4
2
1
1
3
3
3
2
3
2
3
3
3
2
3
3
1
2
4
3
3
4
2
3
2
3
4
4
4
3
1
3
1
3
1
2
4
2
3
2
3
3
3
3
3
2
1
2
2.64
9
1
1
2.06
8
2
2
3.16
2
3
2
2.58
1
2
2
2.79
7
2
3
3.17
6
3
3
2.54
1
2
2
2.91
9
2
2
2.25
7
4
2
2.85
1
3
3
2.86
5
4
3
3.08
1
4
2
2.94
6
4
2
2.87
8
1
2
2.39
2
Appendix C
Opinions about the benefial and harmful effects of multiple-choice questions
1. Beneficial: I don't need to study hard because I just need to read the material.
2. I think MC is easy to do, but it doesn't good way to measure students' understanding.
3. Beneficial: Practice, save time. Harmful: Reduce to give our own answer.
4. Make me more easier to answer questions.
5. 6. Multiple-choice questions decrease the anxiety. If I can't answer the question, I can pick one of the answer randomly.
7. We can save a lot of time to check our answer again and we can increase our ability with new vocabulary.
8. That can help me to remember the answer and elimination wrong answers. It has bad effect because easy to cheat.
9. Multiple-choice is needed to save the time in doing the test because sometimes we don't have enough time to do many questions.
Multiple-choice can also decrease our ambiguity. We can compare the answer from A until E and decide which one will be the best.
But, sometimes multiple-choice can confuse us too if the option almost same.
10. The multiple-choice most have some key word of the answered from the question. And make it more easier to understand.
11. If we do multiple-choice test sometimes we confused about the answer because there are many choices that may be we think
that it was the best answer.
12. For the MC question, it's hard to choose if there is misstyping on its answer.
13. 14. If don't have any idea about the answer, we can choose one of the choices.
15. Multiple-choice questions make student easier to cheat.
16. 17. Sometimes, in some subject, multiple-choice is the best way to answer the question. Multip