Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.79.4.217-224
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Overconfidence and the Performance of Business
Students on Examinations
Paul Sergius Koku & Anique Ahmed Qureshi
To cite this article: Paul Sergius Koku & Anique Ahmed Qureshi (2004) Overconfidence and the
Performance of Business Students on Examinations, Journal of Education for Business, 79:4,
217-224, DOI: 10.3200/JOEB.79.4.217-224
To link to this article: http://dx.doi.org/10.3200/JOEB.79.4.217-224
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 24
View related articles
Citing articles: 6 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:24
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Overconfidence and the
Performance of Business Students
on Examinations
PAUL SERGIUS KOKU
Florida Atlantic University
Fort Lauderdale, Florida
B
ecause our society emphasizes
meritocracy and fairness, a student’s performance on objective and
unbiased examinations has far-reaching
consequences. For example, a high
school student’s performance on the
SAT plays a major role in his or her
acceptance at particular universities or
colleges. Similarly, a business school
student’s performance on the GMAT
will help decide which MBA programs
will accept him or her. Furthermore, a
student’s performance on examinations
while in college generally determines
the student’s course grades, which collectively determine the student’s GPA,
which often serves as an important
merit criterion.
Because of the heavy reliance on a
candidate’s examination performance, a
significant amount of research has
focused on examination instruments and
factors that affect students’ performance.
Studies of performance on cognitive
examinations have shown that, among
other things, a candidate’s performance
is influenced by subgroup differences
(Bobko, Roth, & Potoksy, 1999), environmental test bias (Reynolds & Brown,
1984), biological and genetic factors
(Jensen, 1985), the location of the correct answer in a choice of answers (BarHillel & Attali, 2002), and the test
taker’s confidence (Koriat, Lichtenstein,
& Fischhoff, 1980).
ANIQUE AHMED QURESHI
University of Tampa
Tampa, Florida
ABSTRACT. In this study, the
authors investigated the role of overconfidence on students’ performance
on multiple-choice examinations. The
authors examined the difference
between students’ self-assessed probability of selecting each correct answer
and the proportion of correct answers
that they actually selected. Results
show that high-performing students
discriminated better between difficult
and easy multiple-choice questions
than did low-performing students.
Poorer students exhibited significant
overconfidence with difficult questions. Generally, students displayed
underconfidence with easy questions
and performed better on multiplechoice questions when they were
asked to provide reasons contradicting
their choices. These results suggest
that students can be trained to perform
better on standard multiple-choice
exams through preparation involving
their providing reasons contradicting
their choice of answers.
The effect of students’ confidence on
test performance, especially on multiplechoice examinations, is of particular
interest to us because most professors
who rely on frequent use of multiplechoice examinations to test students have
encountered a fair share of overconfident
students. After taking a multiple-choice
examination, such students often declare
that it was easy and that they would
either “ace” it or do well, only for the
professor to discover that those confident
students did not even pass the examina-
tion. In other words, their declaration
was not in accord with their performance. Are such student declarations
empty bravado designed to impress the
professor or a product of genuine mistaken assessment by the student?
Although it is possible that some of
those declarations are meant to
impress, it is equally possible that some
of them, if not most, are a result of
overconfidence.
Although this is an intriguing phenomenon that has direct implications on
how students and thoughts are evaluated, to the best of our knowledge no
study to date has investigated the effect
of confidence on the performance of
business students on examinations.
Given the large differences among disciplines and the fact that different disciplines attract different personality types,
it is possible that students in one discipline do not behave as those in another
discipline. Thus, in this study we sought
to investigate the role of confidence in
the performance of business students on
examinations—specifically multiplechoice examinations.
Multiple-choice examinations are a
common means of evaluating the
knowledge base of business students.
An investigation of how well the testing
method assesses students’ true knowledge would be useful for several reasons. First, it will provide professors
March/April 2004
217
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
with an insight into a popular evaluative
instrument. Second, understanding the
reasons for students’ performance on
these tests will be useful in shaping the
ways in which instructors teach and
evaluate.
Overconfidence, defined as the unwarranted belief in the accuracy of one’s
judgments (Lichtenstein, Fischhoff, &
Phillips, 1982), has been the subject of
several studies in psychology. Overconfidence could result from the tendency
of an individual to seek confirmatory
evidence while ignoring contradictory
information. We argue that confidence,
or lack of it, is relevant to performance
on examinations, particularly multiplechoice examinations, because an overconfident student, according to the theory that individuals seek confirmatory
evidence while searching memory to
retrieve information (Arkes, Dawes, &
Christensen, 1986; Hoch, 1985; Koriat
et al., 1980), may feel that he or she
already knows the correct answer.
Because of this “feeling,” the individual
may not see the need to search thoroughly among the given alternatives for
the less obvious but correct answer.
Consequently, this tendency will lead to
poor performance.
Studies in psychology on judgment
and overconfidence have shown that
people tend to be overconfident when
estimating the probability that their
judgments are correct (Lichtenstein et
al., 1982). For example, the average
estimate provided by individuals who
were asked to estimate the probability
that they selected the correct answers on
a multiple-choice examination was
found to be consistently greater than the
percentage of questions that they
answered correctly. Very often, an overconfident individual who estimates that
his or her answers to multiple-choice
questions have a 90% chance of being
correct will find that far less than 90%
of the answers are actually correct
(Lichtenstein et al., 1982). However, if
the individual is well calibrated, his or
her confidence would be in accord with
the level of actual performance.
If overconfidence were simply the
result of mistranslating an individual’s
subjective beliefs into numbers, one
would expect that individuals would
show underconfidence as well as over218
Journal of Education for Business
confidence. However, previous studies
have shown that individuals are generally overconfident rather than underconfident (Koriat et al., 1980; Lichtenstein et
al., 1982; Lichtenstein & Fischhoff,
1977; Sniezek, Paese, & Switzer, 1990;
Trafimow & Sniezek, 1994; Wright,
1982). Although this issue is interesting,
the question of why individuals tend to
err mostly on the side of overconfidence
falls within the domain of psychology
and outside the scope of this study.
It has been argued that the basis of
overconfidence lies in an individual’s
tendency or bias to seek confirmatory
evidence while searching memory to
retrieve information. Studies on the
relationship between overconfidence
and performance on multiple-choice
questions have shown that performance
correlates highly with the extent to
which the test taker searched for evidence when considering alternatives
(Koriat et al., 1980). Individuals who
fully considered each alternative among
a given set of choices were better calibrated than individuals who tended to
seek confirming evidence for one alternative. The implication of this finding is
that students who show overconfidence
may be biasing their search toward a
tentatively preferred answer and thus
may not be evaluating fully other alternatives. Thus, it is possible to improve
students’ performance on multiplechoice examinations by making them
develop better calibration through the
required careful consideration of all
alternative answers.
Literature Review
Previous studies have shown that
overconfidence could be caused in three
different ways. First, overconfidence
could result from mistranslating one’s
subjective beliefs into numbers. Second,
overconfidence could be exhibited during the postdecisional stage. During this
stage, an individual who is in the
process of making an overt choice
increases his or her personal commitment to a selected answer. Third, overconfidence could be exhibited during
memory search and retrieval stage. During this process, the individual initially
selects a tentatively preferred answer.
Thereafter, the individual slowly biases
his or her search by focusing on evidence that tends to confirm that initial
selection. The tendency to seek confirmatory evidence increases the individual’s confidence in his or her answers.
These three processes, however, need
not be mutually exclusive (see Koriat et
al., 1980; Sniezek et al., 1990).
We argue that if overconfidence is
simply a matter of mistranslating an
individual’s subjective beliefs into numbers, then one must expect individuals
to exhibit underconfidence as well as
overconfidence. However, the predominant finding of the previous studies is
that individuals generally exhibit overconfidence and not underconfidence.
This absence of underconfidence suggests that miscalibration could not be
solely a result of the fact that individuals simply mistranslate their subjective
beliefs into numbers. Furthermore, previous studies also have shown that overconfidence could not be the result of an
individual’s personal commitment to a
chosen alternative during the postdecisional stage. Instead, overconfidence
was found to result from the tendency to
seek confirmatory evidence in the memory search and retrieval stage (Sniezek
et al., 1990). If that is the case, then it
would seem that the tendency to be
overconfident could be reduced.
Previous attempts at reducing overconfidence have had only marginal
success. Arkes, Christensen, Lai, and
Blumer (1987), in one of the early
attempts to reduce overconfidence,
conducted two experiments. In the first
experiment, the researchers provided
the subjects with feedback on their
performance. The subjects in the second experiment were informed that
they would have to justify their
answers in a group discussion after
they answered the questions. Both
techniques were somewhat successful
in reducing overconfidence.
However, in an earlier study, Koriat et
al. (1980) achieved reasonable success in
reducing overconfidence by asking individuals to list reasons why their answers
might be incorrect. The results from that
study were particularly insightful and
indicated that people tend to be overconfident because they generally focus on
positive rather than negative evidence.
The overconfident individual seemed to
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
consider reasons supporting rather than
contradicting his or her tentatively preferred answer. Because overconfidence
results from concentrating on only the
positive, an effective way to force an
individual to consider other alternatives
carefully is to require him or her to provide possible reasons why his or her
chosen answer could be wrong.
In this study, we extend previous
studies in several ways. Because the
focus of previous studies was on documenting the existence of overconfidence
and analyzing its underlying processes,
those studies did not analyze specifically the effect of overconfidence on performance. Furthermore, because previous researchers generally used very
difficult general knowledge questions to
investigate overconfidence, these studies provided little or no room for
improvement in performance through
deliberation. In such situations, the individuals either knew the answer or they
did not. The role of deductive reasoning
and analysis of such questions was thus
limited.
Our study is different from previous
studies in that our orientation is toward
application of current understanding of
overconfidence. We eliminated the limitation of previous studies by evaluating
overconfidence through (a) coursespecific materials, (b) the use of questions whose levels of difficulty were predetermined by the authors of the textbook that we used (i.e., the questions
were taken from a test bank that was prepared by the authors of the textbook and
published by the publishers of the textbook), and (c) relating overconfidence to
the performance of business students on
multiple-choice examinations. This
study also extends Qureshi (1998) and
Koku and Qureshi (1999) by including
business students in general, regardless
of declared major.
Hypotheses
On the basis of previous studies
(Arkes et al., 1987; Koriat et al., 1980;
Lichtenstein & Fischhoff, 1977), we
hypothesized that a student’s confidence
and judgment on a multiple-choice
examination are influenced by the difficulty level of the examination questions,
the student’s performance, and the stu-
dent’s reasoning mode (i.e., whether the
student provided reasons contradicting
or supporting the multiple-choice
answers). Using previous studies for
guidance, we proposed several hypotheses for testing the effect of overconfidence of students on their performance
on multiple-choice examinations.
We formulated our first hypothesis
(H1) to test the effect of question difficulty on students’ overconfidence. Generally, students who know their material
well should be able to discriminate
between easy and difficult questions.
All other things being equal, one would
expect to see an indirect relationship
between question difficulty and confidence; that is, students who are very
familiar with the subject matter should
exhibit greater confidence with easy
questions and less confidence with difficult questions. However, extrapolations
from previous studies lead to a counterintuitive result. Given what is known
from previous studies, one would expect
that students will tend to exhibit overconfidence with difficult questions
rather than with easy questions. Why?
This will happen because, when taking
multiple-choice examinations, most students tend to either underestimate the
level of difficulty of difficult questions
or fail to appreciate fully the true importance of difficult questions. Thus, we
formulated our first hypothesis:
H1: Overconfidence will increase
as the level of difficulty of questions
increases.
Through the second hypothesis (H2),
we considered the effect of student performance on confidence. Because the
better students (high performing) are
better calibrated than poorer (low performing) students, the former should
discriminate better between what they
know and what they do not know. Thus,
the high performing students will exhibit lower overconfidence with difficult
questions than would low performing
students. In plain language, overconfidence seems to result from a false sense
of “security,” which poorer students
may indulge in. Because low performing students are poorly calibrated, they
would be less discriminating and, unlike
high performing students, seek confirmatory evidence that falsely increases
their confidence. Thus, we formulated
our second hypothesis:
H2: Overconfidence will increase
as the student’s performance decreases.
Through our third and fourth
hypotheses (H3 and H4), we examined
how overconfidence could be reduced.
Specifically, we examined the effect of
asking students to provide reasons that
contradict or support their choice of
answers.
On the basis of Koriat, et al. (1980),
we argue that students realize the limitations of their knowledge when they
are placed in a situation that forces them
to reexamine their choices critically.
Such an environment could be created
when students are required to provide
possible reasons why their answers
could be wrong, along with their chosen
answers. By critically reexamining their
choices, the students would realize that
a question that they initially thought
was easy was not easy after all. We formulated our third hypothesis:
H3: Overconfidence
decreases
when students are required to provide
reasons contradicting their multiplechoice answers.
Furthermore, because requiring students to provide possible contradictory
reasons why their answers could be
wrong forces them to appreciate the full
import of a question, they consequently
would develop a realistic level of appreciation for questions that they initially
thought were easy. On the other hand,
asking students to provide reasons supporting the choice of a multiple-choice
answer allows them to escape from
carefully examining evidence that contradicts the choice. As such, they will
become overconfident with their choice
of answers whether these answers are
correct or not. Thus, we came up with
our fourth hypothesis:
H4: Overconfidence increases when
students are required to provide reasons
supporting their multiple-choice answers.
We formulated our fifth and sixth
hypotheses (H5 and H6) to examine
whether students’ performance on multiple-choice examinations could be
enhanced through a test format.
Specifically, these hypotheses test the
March/April 2004
219
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
relationship between students’ performance and asking students to provide
reasons that support, or reasons that
contradict, their choice of answers.
One would expect that performance
would improve as overconfidence
decreases, because a decrease in confidence with regard to how a student has
performed on an examination suggests
that the student appreciates the “true”
difficulty of questions and does not have
a false sense of security regarding his or
her performance. If requiring students
to provide reasons contradicting their
choice of answers to multiple-choice
questions reduces overconfidence (as
stated in H3), then performance can be
expected to improve as the student considers all the available evidence.
When forced to seriously consider
possible contradictory evidence, a student is less likely to err by selecting the
initially preferred, inaccurate answer;
hence performance will improve. On the
other hand, asking students to provide
reasons supporting their choice of
answers to multiple-choice questions
will increase the students’ overconfidence (as stated in H4). Because the student is not likely to consider all the
available evidence, the student’s performance will suffer. We therefore formulated the following hypotheses:
H5: Students’ performance on
multiple-choice examinations will
improve when they are required to provide possible reasons that contradict
their chosen answers.
H6: Students’ performance on
multiple-choice examinations will suffer when they are required to provide
possible reasons that support their chosen answers.
Method
We collected data from 91 university
undergraduate seniors and juniors majoring in business. These students voluntarily participated in the study; however, all
those who participated in the study were
given five extra points toward their
course grade. We used the final examination in a capstone business course taken
by all business majors, regardless of
major, as an instrument for the study.
The examination questions were
220
Journal of Education for Business
taken from a test bank that was written
by the authors of the textbook used in
the course and published by the textbook publishers. The examination consisted of 50 multiple-choice questions,
and each question had five possible
answers. These questions already were
categorized into three levels of difficulty (easy, intermediate, and difficult) by
the authors of the textbook, who also
wrote the test bank. The students were
given a maximum of 2 hours to take the
examination. Everyone finished the
examination during the time provided.
Because the examination questions
were based on a textbook that was covered completely by us—the authors of
this study—during the course of the
semester, we did not conduct any independent validity and reliability studies
on the test questions. However, we both
crosschecked questions on the examination to ensure that they were understandable and dealt with topics that
were covered in the course.
Students were randomly assigned to
one of the three treatment groups. Students in treatment group 1 were asked
to provide reasons supporting their
answers (the “supporting reasons”
group). Students in treatment group 2
were asked to provide reasons contradicting their choices (the “contradicting
reasons” group), and students in treatment group 3 were asked to provide no
reason for their answers (the “no reason” group). The students in group 1
were asked to write down at least one
strong reason supporting their choice of
answers for each of the multiple-choice
questions. The students in group 2 were
asked to write down at least one strong
reason why each answer could be
wrong. The students in group 3 served
as the control group and were asked to
not write down any reason for their
answers.
We also asked students to write
down their subjective probability
assessment of selecting the correct
chosen answers. In other words, each
student was required to indicate by
each question how confident he or she
was that the chosen answer was correct. The students were allowed to set
their lowest probability estimate at
20% and increase this estimate by 20%
intervals (because there were five pos-
sible choices) until they reached a
maximum of 100%.
We operationalized overconfidence by
evaluating the students’ subjective probability assessment of getting each answer
right. We computed overconfidence and
underconfidence to be equal to the difference between the mean of the probability responses and the overall proportion of
correct answers. This methodology is
consistent with Koriat et al. (1980). A
positive difference would indicate overconfidence and a negative difference
underconfidence. For example, if the
mean confidence judgment was 90%,
and the mean score of the actual performance on the examination was 80%,
then the subject would be overconfident
by 10%. Similarly, if the mean confidence judgment was 80%, and the mean
score of the actual performance on the
examination was 90%, then the subject
would be underconfident by 10%.
We used the students’ scores on the
final examination to measure their actual performance. We preferred using this
measure of performance over the student’s GPA for two reasons. First, the
GPA, as a composite of students’ performance in all the courses that they
took, is amenable to several confounding variables because students take
courses from several disciplines other
than business. For example, in most universities, students are required to take
courses that include general studies, sciences, mathematics, or the humanities.
Second, the method of testing and evaluating students in other courses, even in
the same discipline, could differ significantly depending on the professor and
could range from a take-home examination to merely a term paper. Because of
these variables, the GPA could not be a
reliable measure of actual performance
for this study.
The students’ scores on the examination that we gave ranged from a low of 30
to a high of 98, with a mean of 66 and
standard deviation of 13. We classified
students in the top third of the actual
scores as high performing students.
Those students scored 74 or higher. Similarly, we classified students in the bottom third of the actual scores as low performing students; they scored 60 or
lower. The remaining students were classified as average performing students.
Results
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
We summarize the results of our
analyses in Tables 1 through 4. In Table
1, we summarize the results of all six
hypotheses. The data in Table 2 are a
summary of the calculated means for
the confidence scores, percentage of
correct answers, and over/underconfidence for each of the experimental conditions. Table 3 is a summary of the calculated averages of overconfidence/
underconfidence scores for each group
analyzed through ANOVA, and Table 4
is the summary of our ANOVA analysis
of the performance data for students
who provided reasons supporting or
contradicting their answers and for
those who did not provide any reasons.
On the whole, our analyses show that
the main effects for performance and
question difficulty were significant at
the p < .001 level (see Table 3). They
also show that the students generally
tended to underestimate the difficulty
level of difficult questions; thus they
became “falsely” overconfident with
regard to their performance. In other
words, because they failed to appreciate
the true difficulty level of difficult exam
TABLE 1. Summarized Results From Testing the Six Hypotheses
Hypothesis
H1: Overconfidence increases with the difficulty level
of exam question.
H2: Overconfidence increases as the students’
performance decreases.
H3: Overconfidence decreases when students are
required to provide reasons contradicting their
multiple-choice answers.
H4: Overconfidence increases when students are
required to provide reasons supporting their
multiple choice answers.
H5: Exam performance decreases when students are
required to provide reasons contradicting their
multiple-choice answers.
H6: Exam performance decreases when students are
required to provide reasons supporting their
multiple-choice answers.
Findings
Sig. of F
Supported
0.001
Supported
0.001
Marginally
supported
0.08
Marginally
supported
0.08
Supported
0.001
Not supported
TABLE 2. Calculated Means for Confidence Scores, Percentage of Correct
Answers, and Over- or Underconfidence for Each of the Three Conditions
Condition
Question difficulty
Easy
Intermediate
Difficult
Performance
Low performing
Average performing
High performing
Reasoning mode
Contradictory reasons
Supporting reasons
No reasons (control)
Confidence
scores (%)
Percentage
correct (%)
Over-/underconfidence
(%)
84
77
73
86
70
37
–2
7
36
73
78
83
51
68
80
22
10
3
77
80
76
72
62
65
5
18
11
questions, they tended to think that they
would do better on these questions than
they actually did. As a consequence,
students’ overconfidence increased with
the difficulty level of examination questions. Similarly, because students discriminated less between answers when
they falsely believed that they already
“knew” the correct answer, overconfidence increased as the students’ performance decreased. H1 and H2 were supported by the results.
Our hypotheses regarding (a) the relationship between overconfidence and
giving reasons contradicting chosen
answers and (b) the relationship between
overconfidence and giving reasons supporting chosen answers (H3 and H4)
were marginally supported by the results
(p < .08; see Table 3). Our results show
that critical analyses of questions apparently helped students appreciate the true
import of questions. Thus, students
could realize why their chosen answers
might be either wrong or correct, if the
examination is structured in a way that
forces them to think critically about the
answers that they proffer.
The results reported in Table 2 are
classified into three categories: question
difficulty, performance, and reasoning
mode. The mean confidence scores on
question difficulty show that students
were more confident with difficult questions than they were with easy ones. For
example, the mean confidence score,
which is the mean of the students’ subjective probability of selecting a correct
response, on easy questions was 84%;
however, the actual percentage of correct responses to the easy questions was
86%. Thus, the students were actually
slightly (2%) underconfident with easy
questions. This is an unexpected finding
and suggests that students were uncertain when they encountered easy questions and perhaps read more into them
than necessary.
The mean confidence score on difficult questions was 73%, whereas the
actual percentage of correct responses
to the difficult questions was 51%.
Thus, the students were grossly (31%)
overconfident with regard to difficult
questions. For questions with an intermediate level of difficulty, the mean
confidence score was 77% and the actual percentage of correct responses was
March/April 2004
221
TABLE 3. ANOVA Analysis of Calculated Averages of Over- and
Underconfidence Scores for Each Group
Source of variations
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Between subjects
P (performance)
R (reasoning mode)
P by R
P by R by S (subject)
Within subject variations
Q (question difficulty)
P by Q
R by Q
P by R by Q
P by R by Q by S
SS
DF
7307.3
1815.61
568.39
28533.3
2
2
4
82
55989.85
482.73
1142
914.32
29136.38
2
4
4
8
164
F
Sig. of F
3653.75
907.8
142.1
347.97
10.5
2.61
0.41
0.001
0.08
0.802
27994.93
120.68
281.11
114.29
177.66
157.58
0.68
1.58
0.64
0.001
0.607
0.181
0.74
MS
TABLE 4. ANOVA Analysis of Calculated Averages of Over- and
Underconfidence Scores for Each Group
Between groups
Within groups
Total
SS
DF
1711.66
14275.2
159986.9
2
88
90
70%, indicating a moderate (7%) level
of overconfidence.
With respect to performance, the
mean probability assessment of high
performing students was 83%, and their
percentage of correct responses was
80%, indicating a slight level of overconfidence (3%). The mean probability
assessment of low performing students
was 73%, whereas their percentage of
correct responses was 51%, indicating
an extreme (22%) level of overconfidence. The average-level-performing
students had a mean probability assessment score of 78%, and their actual performance was 68%. Thus, they were
moderately overconfident (10%) with
their answers.
The results of the “reasoning mode”
of the experiment are interesting and in
accord with our hypothesis. They show
that overconfidence decreased when
students were asked to provide reasons
why their chosen answers could be
wrong. The mean confidence score of
the “contradictory reasons” group was
77%, whereas its percentage of correct
answers was 72%, indicating only a 5%
level of overconfidence. On the other
222
Journal of Education for Business
MS
F
855.83
162.219
5.2758
Sig. of F
0.0069
hand, the mean confidence score of the
“supporting reasons” group was 80%,
whereas its percentage of correct
answers was 62%. This result indicates
an overconfidence level of 18% and
underscores our argument that students
are less discriminating and therefore
more likely to select wrong answers
when they seek evidence to confirm
their initially chosen answers instead of
carefully analyzing all the possible
answers before making a selection.
The mean confidence score of the
control group—the group that did not
provide any reasons for or against their
choice of answers—was 76%, and its
percentage of correct answers was 65%.
This result produces an overconfidence
score of 11%, a fairly significant
amount of overconfidence.
We next investigated whether there
was a significant difference among the
overconfidence levels of the three
groups. To answer this question, we
conducted the Bonferroni multiple comparison test to analyze the differences
among the means of the three groups.
The results suggest that there was a significant difference ( p < .05) among the
three groups. However, there was no
significant difference (p < .8) between
the “supporting reasons” and the “no
reasons” groups.
It is evident from these results that
asking students to provide reasons that
contradict their multiple-choice answers
reduces overconfidence. However, asking students to provide reasons that support their multiple-choice answers does
not increase their overconfidence. On
the other hand, the overconfidence
scores were about the same for those
who provided supporting reasons and
for those who provided no reason. We
note that the supporting and the contradicting reasons provided by students
varied widely in quality; however, an
exploratory analysis of the reasons
revealed no meaningful relationship.
In Table 4, we provide one-way
ANOVA results of our analysis of (a)
the relationship between students’ performance and asking them to provide
reasons that support their choice of
answers and (b) the relationship
between performance and asking students to provide reasons that contradict
their choice of answers (H5 and H6).
Because .0069 is less that .05, the null
hypothesis of equal means of performance level is rejected. In other words,
there was a statistically significant difference in students’ performance levels
between the group asked to provide reasons in support of its choices and the
group asked to contradict its choices.
The results reported in Tables 2 and 4
support our hypothesis that asking students to provide reasons supporting
their chosen answers causes them to
miss the chance to confront the possibility that they could be selecting the
wrong answer. Asking them to provide
reasons that contradict their answers not
only forces them to think critically
about their initial choices, but it also
gives them a second opportunity to
change their initially chosen answers if
the reason(s) for their initial selections
are found to be nonconvincing or less
logical.
Discussion
The findings of this study represent
the preliminary results of our attempt to
understand the effect of overconfidence
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
of business students on their performance on multiple-choice examinations.
Even though we used the multiplechoice format in this study, we believe
that the results could be generalized to
all types of examinations. We attempted to reduce overconfidence and
improve students’ performance on
multiple-choice examinations by asking
them to provide reasons contradicting or
supporting their chosen answers.
The results provide strong evidence
that high performing students appropriately exhibit more confidence in their
answers. High performing students are
also better calibrated and exhibit lower
levels of overconfidence. These students
are able to better discriminate between
what they know and what they do not
know. In contrast, although low performing students appropriately exhibit
lower confidence in their answers, they
are poorly calibrated. This suggests that
although high performing students are
more likely to recognize the extent and
limitation of their knowledge, low performing students have only limited
insight into their performance.
In general, students appropriately
exhibit greater confidence with easy
questions and lower confidence with
difficult questions. However, the students in this study seemed to be generally poorly calibrated with regard to difficult questions, with which they
demonstrated a significant level of overconfidence. This tendency could indicate that, on the whole, students do not
seem to appreciate the difficulty level of
the examination questions. One unexpected finding of the study is that the
students actually exhibited a slight
underconfidence with easy questions,
which suggests that they were uncertain
when encountering easy questions.
Thus, they may have attempted to read
more into the (easy) questions than was
actually warranted or necessary. Perhaps they thought that the easy questions were “trick” questions.
The results also show that asking
students to provide reasons that contradict their multiple-choice answers is
successful in reducing overconfidence
and improving student performance on
multiple-choice examinations. This
strategy seems to force them to study
more carefully the alternative answers
that they had initially rejected. Furthermore, it would seem that by concentrating on the weakness of their answers,
the students overtly recognize the limitations of their abilities, which results in
their becoming better calibrated and
thus exhibiting lower overconfidence.
In contrast, the results further show
that asking students to provide reasons
supporting
their
multiple-choice
answers did slightly increase overconfidence. However, we observed no statistically significant difference between
the group that provided reasons supporting their answers and the group that provided no reasons. This finding suggests
that students were already focusing on
reasons supporting their answers; as
such, asking them to provide supporting
reasons did not have a significant
impact.
Consistent with previous research in
psychology, we found that business students were overconfident when estimating the probability that their answers to
multiple-choice questions would be correct. The results of this study indicate
that instructors can remedy this false
sense of security that students demonstrate on multiple-choice examinations
by changing the format of multiplechoice examinations to require that the
student must not only provide the correct answer but also indicate why the
correct answer is correct or the incorrect
answer is incorrect. We contend that this
format not only will improve students’
performance but also will contribute
indirectly to learning, as it will help
make the student better calibrated and
thus less overconfident. A student who
is not overconfident will more likely
appreciate his or her lack of depth on
the subject matter and seek to remedy
the situation by studying.
Other equally interesting areas of
future research that could influence the
way that we evaluate students should
include the effect of question format
and presentation order on students’ confidence and performance on examinations. It would be interesting to see if
the order of presenting questions in
terms of difficulty (e.g., easy questions
followed by difficult ones) has any significant impact on students’ confidence
and thus their performance. An argument could be made that if students
encounter easy questions first, they
might become overconfident and their
exam performance might suffer. On the
other hand, if examination questions are
ordered from difficult to easy, students
might “panic” and the resulting performance might suffer also. Future
research on this issue could provide
invaluable insight for professors on how
best to evaluate students.
Furthermore, future research on other
methods of reducing overconfidence,
such as using computerized testing
methods that provide students with
immediate feedback on their performance on one question before they
move on to the next, might be interesting and worthwhile. Future research
could also focus on the effect of time
constraints on confidence, overconfidence, and performance.
The results of this study could have
significant implications, given the widespread use of the multiple-test format in
business education. The results indicate
that students indeed can be trained to
perform better on the standard multiplechoice exams—such as the Certified
Public Accountant (CPA) and Certified
Financial Analyst (CFA) exams—when
they are prepared through multiplechoice exams that ask them to provide
reasons contradicting their answer
selections.
REFERENCES
Arkes, H. R., Dawes, R. M., & Christensen, C.
(1986). Factors influencing the use of decision
rule in a probabilistic task. Organizational
Behavior and Human Decision Processes, 37,
93–110.
Arkes, H. R., Christensen, C., Lai, C., & Blumer,
C. (1987). Two methods of reducing overconfidence. Organizational Behavior and Human
Decision Processes, 39, 133–144.
Bar-Hillel, M., & Attali, Y. (2002). “The delicate
art of key balancing.” Or: When randomization
is too important to be trusted to chance. The
American Statistician, 564, 299–305.
Bobko, P., Roth, P. L., & Potosky, D. (1999).
Derivation and implications of a meta-analytic
matrix incorporating cognitive ability, alternative predictors, and job performance. Personnel
Psychology, 52, 561–589.
Hoch, S. J. (1985). Counterfactual reasoning and
accuracy in predicting personal events. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 11, 719–731.
Jensen, A. R. (1985). The nature of Black-White
difference on various psychometric tests:
Spearman’s hypothesis. Behavioral and Brain
Sciences, 8, 193–263.
Koku, P. S., & Qureshi, A. A. (1999). Does confidence impact the performance on multiplechoice examinations? The case of marketing
March/April 2004
223
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
students. Academy of Marketing Science Congress, Malta.
Koriat, A., Lichtenstein, S., & Fischhoff, B.
(1980). Reasons for confidence. Journal of
Experimental Psychology: Human Learning
and Memory, 6, 107–118.
Lichtenstein, S., & Fischhoff, B. (1977). Do those
who know more also know more about how
much they know? Organizational Behavior and
Human Decision Processes, 20, 159–183.
Lichtenstein, S., Fischhoff, B., & Phillips, L. D.
224
Journal of Education for Business
(1982). Calibration of probabilities: The state
of the art to 1980. In D. Kahneman, P. Slovic, &
A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
Qureshi, A. A. (1998). Preliminary evidence about
the effect of confidence on exam performance.
American Academy of Accounting and
Finance, December.
Reynolds, C., & Brown, R. (1984). Perspectives
on bias in mental testing. New York: Plenum.
Sniezek, J., Paese, P., & Switzer, F., III. (1990).
The effect of choosing on confidence in choice.
Organizational Behavior and Human Decision
Processes, 46, 264–282.
Trafimow, D., & Sniezek, J. (1994). Perceived
performing and its effect on confidence. Organizational Behavior and Human Decision
Processes, 57, 290–302.
Wright, G. (1982). Changes in realism of probability assessments as a function of question
type. Acta Psychologica, 52, 165–174.
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Overconfidence and the Performance of Business
Students on Examinations
Paul Sergius Koku & Anique Ahmed Qureshi
To cite this article: Paul Sergius Koku & Anique Ahmed Qureshi (2004) Overconfidence and the
Performance of Business Students on Examinations, Journal of Education for Business, 79:4,
217-224, DOI: 10.3200/JOEB.79.4.217-224
To link to this article: http://dx.doi.org/10.3200/JOEB.79.4.217-224
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 24
View related articles
Citing articles: 6 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:24
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Overconfidence and the
Performance of Business Students
on Examinations
PAUL SERGIUS KOKU
Florida Atlantic University
Fort Lauderdale, Florida
B
ecause our society emphasizes
meritocracy and fairness, a student’s performance on objective and
unbiased examinations has far-reaching
consequences. For example, a high
school student’s performance on the
SAT plays a major role in his or her
acceptance at particular universities or
colleges. Similarly, a business school
student’s performance on the GMAT
will help decide which MBA programs
will accept him or her. Furthermore, a
student’s performance on examinations
while in college generally determines
the student’s course grades, which collectively determine the student’s GPA,
which often serves as an important
merit criterion.
Because of the heavy reliance on a
candidate’s examination performance, a
significant amount of research has
focused on examination instruments and
factors that affect students’ performance.
Studies of performance on cognitive
examinations have shown that, among
other things, a candidate’s performance
is influenced by subgroup differences
(Bobko, Roth, & Potoksy, 1999), environmental test bias (Reynolds & Brown,
1984), biological and genetic factors
(Jensen, 1985), the location of the correct answer in a choice of answers (BarHillel & Attali, 2002), and the test
taker’s confidence (Koriat, Lichtenstein,
& Fischhoff, 1980).
ANIQUE AHMED QURESHI
University of Tampa
Tampa, Florida
ABSTRACT. In this study, the
authors investigated the role of overconfidence on students’ performance
on multiple-choice examinations. The
authors examined the difference
between students’ self-assessed probability of selecting each correct answer
and the proportion of correct answers
that they actually selected. Results
show that high-performing students
discriminated better between difficult
and easy multiple-choice questions
than did low-performing students.
Poorer students exhibited significant
overconfidence with difficult questions. Generally, students displayed
underconfidence with easy questions
and performed better on multiplechoice questions when they were
asked to provide reasons contradicting
their choices. These results suggest
that students can be trained to perform
better on standard multiple-choice
exams through preparation involving
their providing reasons contradicting
their choice of answers.
The effect of students’ confidence on
test performance, especially on multiplechoice examinations, is of particular
interest to us because most professors
who rely on frequent use of multiplechoice examinations to test students have
encountered a fair share of overconfident
students. After taking a multiple-choice
examination, such students often declare
that it was easy and that they would
either “ace” it or do well, only for the
professor to discover that those confident
students did not even pass the examina-
tion. In other words, their declaration
was not in accord with their performance. Are such student declarations
empty bravado designed to impress the
professor or a product of genuine mistaken assessment by the student?
Although it is possible that some of
those declarations are meant to
impress, it is equally possible that some
of them, if not most, are a result of
overconfidence.
Although this is an intriguing phenomenon that has direct implications on
how students and thoughts are evaluated, to the best of our knowledge no
study to date has investigated the effect
of confidence on the performance of
business students on examinations.
Given the large differences among disciplines and the fact that different disciplines attract different personality types,
it is possible that students in one discipline do not behave as those in another
discipline. Thus, in this study we sought
to investigate the role of confidence in
the performance of business students on
examinations—specifically multiplechoice examinations.
Multiple-choice examinations are a
common means of evaluating the
knowledge base of business students.
An investigation of how well the testing
method assesses students’ true knowledge would be useful for several reasons. First, it will provide professors
March/April 2004
217
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
with an insight into a popular evaluative
instrument. Second, understanding the
reasons for students’ performance on
these tests will be useful in shaping the
ways in which instructors teach and
evaluate.
Overconfidence, defined as the unwarranted belief in the accuracy of one’s
judgments (Lichtenstein, Fischhoff, &
Phillips, 1982), has been the subject of
several studies in psychology. Overconfidence could result from the tendency
of an individual to seek confirmatory
evidence while ignoring contradictory
information. We argue that confidence,
or lack of it, is relevant to performance
on examinations, particularly multiplechoice examinations, because an overconfident student, according to the theory that individuals seek confirmatory
evidence while searching memory to
retrieve information (Arkes, Dawes, &
Christensen, 1986; Hoch, 1985; Koriat
et al., 1980), may feel that he or she
already knows the correct answer.
Because of this “feeling,” the individual
may not see the need to search thoroughly among the given alternatives for
the less obvious but correct answer.
Consequently, this tendency will lead to
poor performance.
Studies in psychology on judgment
and overconfidence have shown that
people tend to be overconfident when
estimating the probability that their
judgments are correct (Lichtenstein et
al., 1982). For example, the average
estimate provided by individuals who
were asked to estimate the probability
that they selected the correct answers on
a multiple-choice examination was
found to be consistently greater than the
percentage of questions that they
answered correctly. Very often, an overconfident individual who estimates that
his or her answers to multiple-choice
questions have a 90% chance of being
correct will find that far less than 90%
of the answers are actually correct
(Lichtenstein et al., 1982). However, if
the individual is well calibrated, his or
her confidence would be in accord with
the level of actual performance.
If overconfidence were simply the
result of mistranslating an individual’s
subjective beliefs into numbers, one
would expect that individuals would
show underconfidence as well as over218
Journal of Education for Business
confidence. However, previous studies
have shown that individuals are generally overconfident rather than underconfident (Koriat et al., 1980; Lichtenstein et
al., 1982; Lichtenstein & Fischhoff,
1977; Sniezek, Paese, & Switzer, 1990;
Trafimow & Sniezek, 1994; Wright,
1982). Although this issue is interesting,
the question of why individuals tend to
err mostly on the side of overconfidence
falls within the domain of psychology
and outside the scope of this study.
It has been argued that the basis of
overconfidence lies in an individual’s
tendency or bias to seek confirmatory
evidence while searching memory to
retrieve information. Studies on the
relationship between overconfidence
and performance on multiple-choice
questions have shown that performance
correlates highly with the extent to
which the test taker searched for evidence when considering alternatives
(Koriat et al., 1980). Individuals who
fully considered each alternative among
a given set of choices were better calibrated than individuals who tended to
seek confirming evidence for one alternative. The implication of this finding is
that students who show overconfidence
may be biasing their search toward a
tentatively preferred answer and thus
may not be evaluating fully other alternatives. Thus, it is possible to improve
students’ performance on multiplechoice examinations by making them
develop better calibration through the
required careful consideration of all
alternative answers.
Literature Review
Previous studies have shown that
overconfidence could be caused in three
different ways. First, overconfidence
could result from mistranslating one’s
subjective beliefs into numbers. Second,
overconfidence could be exhibited during the postdecisional stage. During this
stage, an individual who is in the
process of making an overt choice
increases his or her personal commitment to a selected answer. Third, overconfidence could be exhibited during
memory search and retrieval stage. During this process, the individual initially
selects a tentatively preferred answer.
Thereafter, the individual slowly biases
his or her search by focusing on evidence that tends to confirm that initial
selection. The tendency to seek confirmatory evidence increases the individual’s confidence in his or her answers.
These three processes, however, need
not be mutually exclusive (see Koriat et
al., 1980; Sniezek et al., 1990).
We argue that if overconfidence is
simply a matter of mistranslating an
individual’s subjective beliefs into numbers, then one must expect individuals
to exhibit underconfidence as well as
overconfidence. However, the predominant finding of the previous studies is
that individuals generally exhibit overconfidence and not underconfidence.
This absence of underconfidence suggests that miscalibration could not be
solely a result of the fact that individuals simply mistranslate their subjective
beliefs into numbers. Furthermore, previous studies also have shown that overconfidence could not be the result of an
individual’s personal commitment to a
chosen alternative during the postdecisional stage. Instead, overconfidence
was found to result from the tendency to
seek confirmatory evidence in the memory search and retrieval stage (Sniezek
et al., 1990). If that is the case, then it
would seem that the tendency to be
overconfident could be reduced.
Previous attempts at reducing overconfidence have had only marginal
success. Arkes, Christensen, Lai, and
Blumer (1987), in one of the early
attempts to reduce overconfidence,
conducted two experiments. In the first
experiment, the researchers provided
the subjects with feedback on their
performance. The subjects in the second experiment were informed that
they would have to justify their
answers in a group discussion after
they answered the questions. Both
techniques were somewhat successful
in reducing overconfidence.
However, in an earlier study, Koriat et
al. (1980) achieved reasonable success in
reducing overconfidence by asking individuals to list reasons why their answers
might be incorrect. The results from that
study were particularly insightful and
indicated that people tend to be overconfident because they generally focus on
positive rather than negative evidence.
The overconfident individual seemed to
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
consider reasons supporting rather than
contradicting his or her tentatively preferred answer. Because overconfidence
results from concentrating on only the
positive, an effective way to force an
individual to consider other alternatives
carefully is to require him or her to provide possible reasons why his or her
chosen answer could be wrong.
In this study, we extend previous
studies in several ways. Because the
focus of previous studies was on documenting the existence of overconfidence
and analyzing its underlying processes,
those studies did not analyze specifically the effect of overconfidence on performance. Furthermore, because previous researchers generally used very
difficult general knowledge questions to
investigate overconfidence, these studies provided little or no room for
improvement in performance through
deliberation. In such situations, the individuals either knew the answer or they
did not. The role of deductive reasoning
and analysis of such questions was thus
limited.
Our study is different from previous
studies in that our orientation is toward
application of current understanding of
overconfidence. We eliminated the limitation of previous studies by evaluating
overconfidence through (a) coursespecific materials, (b) the use of questions whose levels of difficulty were predetermined by the authors of the textbook that we used (i.e., the questions
were taken from a test bank that was prepared by the authors of the textbook and
published by the publishers of the textbook), and (c) relating overconfidence to
the performance of business students on
multiple-choice examinations. This
study also extends Qureshi (1998) and
Koku and Qureshi (1999) by including
business students in general, regardless
of declared major.
Hypotheses
On the basis of previous studies
(Arkes et al., 1987; Koriat et al., 1980;
Lichtenstein & Fischhoff, 1977), we
hypothesized that a student’s confidence
and judgment on a multiple-choice
examination are influenced by the difficulty level of the examination questions,
the student’s performance, and the stu-
dent’s reasoning mode (i.e., whether the
student provided reasons contradicting
or supporting the multiple-choice
answers). Using previous studies for
guidance, we proposed several hypotheses for testing the effect of overconfidence of students on their performance
on multiple-choice examinations.
We formulated our first hypothesis
(H1) to test the effect of question difficulty on students’ overconfidence. Generally, students who know their material
well should be able to discriminate
between easy and difficult questions.
All other things being equal, one would
expect to see an indirect relationship
between question difficulty and confidence; that is, students who are very
familiar with the subject matter should
exhibit greater confidence with easy
questions and less confidence with difficult questions. However, extrapolations
from previous studies lead to a counterintuitive result. Given what is known
from previous studies, one would expect
that students will tend to exhibit overconfidence with difficult questions
rather than with easy questions. Why?
This will happen because, when taking
multiple-choice examinations, most students tend to either underestimate the
level of difficulty of difficult questions
or fail to appreciate fully the true importance of difficult questions. Thus, we
formulated our first hypothesis:
H1: Overconfidence will increase
as the level of difficulty of questions
increases.
Through the second hypothesis (H2),
we considered the effect of student performance on confidence. Because the
better students (high performing) are
better calibrated than poorer (low performing) students, the former should
discriminate better between what they
know and what they do not know. Thus,
the high performing students will exhibit lower overconfidence with difficult
questions than would low performing
students. In plain language, overconfidence seems to result from a false sense
of “security,” which poorer students
may indulge in. Because low performing students are poorly calibrated, they
would be less discriminating and, unlike
high performing students, seek confirmatory evidence that falsely increases
their confidence. Thus, we formulated
our second hypothesis:
H2: Overconfidence will increase
as the student’s performance decreases.
Through our third and fourth
hypotheses (H3 and H4), we examined
how overconfidence could be reduced.
Specifically, we examined the effect of
asking students to provide reasons that
contradict or support their choice of
answers.
On the basis of Koriat, et al. (1980),
we argue that students realize the limitations of their knowledge when they
are placed in a situation that forces them
to reexamine their choices critically.
Such an environment could be created
when students are required to provide
possible reasons why their answers
could be wrong, along with their chosen
answers. By critically reexamining their
choices, the students would realize that
a question that they initially thought
was easy was not easy after all. We formulated our third hypothesis:
H3: Overconfidence
decreases
when students are required to provide
reasons contradicting their multiplechoice answers.
Furthermore, because requiring students to provide possible contradictory
reasons why their answers could be
wrong forces them to appreciate the full
import of a question, they consequently
would develop a realistic level of appreciation for questions that they initially
thought were easy. On the other hand,
asking students to provide reasons supporting the choice of a multiple-choice
answer allows them to escape from
carefully examining evidence that contradicts the choice. As such, they will
become overconfident with their choice
of answers whether these answers are
correct or not. Thus, we came up with
our fourth hypothesis:
H4: Overconfidence increases when
students are required to provide reasons
supporting their multiple-choice answers.
We formulated our fifth and sixth
hypotheses (H5 and H6) to examine
whether students’ performance on multiple-choice examinations could be
enhanced through a test format.
Specifically, these hypotheses test the
March/April 2004
219
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
relationship between students’ performance and asking students to provide
reasons that support, or reasons that
contradict, their choice of answers.
One would expect that performance
would improve as overconfidence
decreases, because a decrease in confidence with regard to how a student has
performed on an examination suggests
that the student appreciates the “true”
difficulty of questions and does not have
a false sense of security regarding his or
her performance. If requiring students
to provide reasons contradicting their
choice of answers to multiple-choice
questions reduces overconfidence (as
stated in H3), then performance can be
expected to improve as the student considers all the available evidence.
When forced to seriously consider
possible contradictory evidence, a student is less likely to err by selecting the
initially preferred, inaccurate answer;
hence performance will improve. On the
other hand, asking students to provide
reasons supporting their choice of
answers to multiple-choice questions
will increase the students’ overconfidence (as stated in H4). Because the student is not likely to consider all the
available evidence, the student’s performance will suffer. We therefore formulated the following hypotheses:
H5: Students’ performance on
multiple-choice examinations will
improve when they are required to provide possible reasons that contradict
their chosen answers.
H6: Students’ performance on
multiple-choice examinations will suffer when they are required to provide
possible reasons that support their chosen answers.
Method
We collected data from 91 university
undergraduate seniors and juniors majoring in business. These students voluntarily participated in the study; however, all
those who participated in the study were
given five extra points toward their
course grade. We used the final examination in a capstone business course taken
by all business majors, regardless of
major, as an instrument for the study.
The examination questions were
220
Journal of Education for Business
taken from a test bank that was written
by the authors of the textbook used in
the course and published by the textbook publishers. The examination consisted of 50 multiple-choice questions,
and each question had five possible
answers. These questions already were
categorized into three levels of difficulty (easy, intermediate, and difficult) by
the authors of the textbook, who also
wrote the test bank. The students were
given a maximum of 2 hours to take the
examination. Everyone finished the
examination during the time provided.
Because the examination questions
were based on a textbook that was covered completely by us—the authors of
this study—during the course of the
semester, we did not conduct any independent validity and reliability studies
on the test questions. However, we both
crosschecked questions on the examination to ensure that they were understandable and dealt with topics that
were covered in the course.
Students were randomly assigned to
one of the three treatment groups. Students in treatment group 1 were asked
to provide reasons supporting their
answers (the “supporting reasons”
group). Students in treatment group 2
were asked to provide reasons contradicting their choices (the “contradicting
reasons” group), and students in treatment group 3 were asked to provide no
reason for their answers (the “no reason” group). The students in group 1
were asked to write down at least one
strong reason supporting their choice of
answers for each of the multiple-choice
questions. The students in group 2 were
asked to write down at least one strong
reason why each answer could be
wrong. The students in group 3 served
as the control group and were asked to
not write down any reason for their
answers.
We also asked students to write
down their subjective probability
assessment of selecting the correct
chosen answers. In other words, each
student was required to indicate by
each question how confident he or she
was that the chosen answer was correct. The students were allowed to set
their lowest probability estimate at
20% and increase this estimate by 20%
intervals (because there were five pos-
sible choices) until they reached a
maximum of 100%.
We operationalized overconfidence by
evaluating the students’ subjective probability assessment of getting each answer
right. We computed overconfidence and
underconfidence to be equal to the difference between the mean of the probability responses and the overall proportion of
correct answers. This methodology is
consistent with Koriat et al. (1980). A
positive difference would indicate overconfidence and a negative difference
underconfidence. For example, if the
mean confidence judgment was 90%,
and the mean score of the actual performance on the examination was 80%,
then the subject would be overconfident
by 10%. Similarly, if the mean confidence judgment was 80%, and the mean
score of the actual performance on the
examination was 90%, then the subject
would be underconfident by 10%.
We used the students’ scores on the
final examination to measure their actual performance. We preferred using this
measure of performance over the student’s GPA for two reasons. First, the
GPA, as a composite of students’ performance in all the courses that they
took, is amenable to several confounding variables because students take
courses from several disciplines other
than business. For example, in most universities, students are required to take
courses that include general studies, sciences, mathematics, or the humanities.
Second, the method of testing and evaluating students in other courses, even in
the same discipline, could differ significantly depending on the professor and
could range from a take-home examination to merely a term paper. Because of
these variables, the GPA could not be a
reliable measure of actual performance
for this study.
The students’ scores on the examination that we gave ranged from a low of 30
to a high of 98, with a mean of 66 and
standard deviation of 13. We classified
students in the top third of the actual
scores as high performing students.
Those students scored 74 or higher. Similarly, we classified students in the bottom third of the actual scores as low performing students; they scored 60 or
lower. The remaining students were classified as average performing students.
Results
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
We summarize the results of our
analyses in Tables 1 through 4. In Table
1, we summarize the results of all six
hypotheses. The data in Table 2 are a
summary of the calculated means for
the confidence scores, percentage of
correct answers, and over/underconfidence for each of the experimental conditions. Table 3 is a summary of the calculated averages of overconfidence/
underconfidence scores for each group
analyzed through ANOVA, and Table 4
is the summary of our ANOVA analysis
of the performance data for students
who provided reasons supporting or
contradicting their answers and for
those who did not provide any reasons.
On the whole, our analyses show that
the main effects for performance and
question difficulty were significant at
the p < .001 level (see Table 3). They
also show that the students generally
tended to underestimate the difficulty
level of difficult questions; thus they
became “falsely” overconfident with
regard to their performance. In other
words, because they failed to appreciate
the true difficulty level of difficult exam
TABLE 1. Summarized Results From Testing the Six Hypotheses
Hypothesis
H1: Overconfidence increases with the difficulty level
of exam question.
H2: Overconfidence increases as the students’
performance decreases.
H3: Overconfidence decreases when students are
required to provide reasons contradicting their
multiple-choice answers.
H4: Overconfidence increases when students are
required to provide reasons supporting their
multiple choice answers.
H5: Exam performance decreases when students are
required to provide reasons contradicting their
multiple-choice answers.
H6: Exam performance decreases when students are
required to provide reasons supporting their
multiple-choice answers.
Findings
Sig. of F
Supported
0.001
Supported
0.001
Marginally
supported
0.08
Marginally
supported
0.08
Supported
0.001
Not supported
TABLE 2. Calculated Means for Confidence Scores, Percentage of Correct
Answers, and Over- or Underconfidence for Each of the Three Conditions
Condition
Question difficulty
Easy
Intermediate
Difficult
Performance
Low performing
Average performing
High performing
Reasoning mode
Contradictory reasons
Supporting reasons
No reasons (control)
Confidence
scores (%)
Percentage
correct (%)
Over-/underconfidence
(%)
84
77
73
86
70
37
–2
7
36
73
78
83
51
68
80
22
10
3
77
80
76
72
62
65
5
18
11
questions, they tended to think that they
would do better on these questions than
they actually did. As a consequence,
students’ overconfidence increased with
the difficulty level of examination questions. Similarly, because students discriminated less between answers when
they falsely believed that they already
“knew” the correct answer, overconfidence increased as the students’ performance decreased. H1 and H2 were supported by the results.
Our hypotheses regarding (a) the relationship between overconfidence and
giving reasons contradicting chosen
answers and (b) the relationship between
overconfidence and giving reasons supporting chosen answers (H3 and H4)
were marginally supported by the results
(p < .08; see Table 3). Our results show
that critical analyses of questions apparently helped students appreciate the true
import of questions. Thus, students
could realize why their chosen answers
might be either wrong or correct, if the
examination is structured in a way that
forces them to think critically about the
answers that they proffer.
The results reported in Table 2 are
classified into three categories: question
difficulty, performance, and reasoning
mode. The mean confidence scores on
question difficulty show that students
were more confident with difficult questions than they were with easy ones. For
example, the mean confidence score,
which is the mean of the students’ subjective probability of selecting a correct
response, on easy questions was 84%;
however, the actual percentage of correct responses to the easy questions was
86%. Thus, the students were actually
slightly (2%) underconfident with easy
questions. This is an unexpected finding
and suggests that students were uncertain when they encountered easy questions and perhaps read more into them
than necessary.
The mean confidence score on difficult questions was 73%, whereas the
actual percentage of correct responses
to the difficult questions was 51%.
Thus, the students were grossly (31%)
overconfident with regard to difficult
questions. For questions with an intermediate level of difficulty, the mean
confidence score was 77% and the actual percentage of correct responses was
March/April 2004
221
TABLE 3. ANOVA Analysis of Calculated Averages of Over- and
Underconfidence Scores for Each Group
Source of variations
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Between subjects
P (performance)
R (reasoning mode)
P by R
P by R by S (subject)
Within subject variations
Q (question difficulty)
P by Q
R by Q
P by R by Q
P by R by Q by S
SS
DF
7307.3
1815.61
568.39
28533.3
2
2
4
82
55989.85
482.73
1142
914.32
29136.38
2
4
4
8
164
F
Sig. of F
3653.75
907.8
142.1
347.97
10.5
2.61
0.41
0.001
0.08
0.802
27994.93
120.68
281.11
114.29
177.66
157.58
0.68
1.58
0.64
0.001
0.607
0.181
0.74
MS
TABLE 4. ANOVA Analysis of Calculated Averages of Over- and
Underconfidence Scores for Each Group
Between groups
Within groups
Total
SS
DF
1711.66
14275.2
159986.9
2
88
90
70%, indicating a moderate (7%) level
of overconfidence.
With respect to performance, the
mean probability assessment of high
performing students was 83%, and their
percentage of correct responses was
80%, indicating a slight level of overconfidence (3%). The mean probability
assessment of low performing students
was 73%, whereas their percentage of
correct responses was 51%, indicating
an extreme (22%) level of overconfidence. The average-level-performing
students had a mean probability assessment score of 78%, and their actual performance was 68%. Thus, they were
moderately overconfident (10%) with
their answers.
The results of the “reasoning mode”
of the experiment are interesting and in
accord with our hypothesis. They show
that overconfidence decreased when
students were asked to provide reasons
why their chosen answers could be
wrong. The mean confidence score of
the “contradictory reasons” group was
77%, whereas its percentage of correct
answers was 72%, indicating only a 5%
level of overconfidence. On the other
222
Journal of Education for Business
MS
F
855.83
162.219
5.2758
Sig. of F
0.0069
hand, the mean confidence score of the
“supporting reasons” group was 80%,
whereas its percentage of correct
answers was 62%. This result indicates
an overconfidence level of 18% and
underscores our argument that students
are less discriminating and therefore
more likely to select wrong answers
when they seek evidence to confirm
their initially chosen answers instead of
carefully analyzing all the possible
answers before making a selection.
The mean confidence score of the
control group—the group that did not
provide any reasons for or against their
choice of answers—was 76%, and its
percentage of correct answers was 65%.
This result produces an overconfidence
score of 11%, a fairly significant
amount of overconfidence.
We next investigated whether there
was a significant difference among the
overconfidence levels of the three
groups. To answer this question, we
conducted the Bonferroni multiple comparison test to analyze the differences
among the means of the three groups.
The results suggest that there was a significant difference ( p < .05) among the
three groups. However, there was no
significant difference (p < .8) between
the “supporting reasons” and the “no
reasons” groups.
It is evident from these results that
asking students to provide reasons that
contradict their multiple-choice answers
reduces overconfidence. However, asking students to provide reasons that support their multiple-choice answers does
not increase their overconfidence. On
the other hand, the overconfidence
scores were about the same for those
who provided supporting reasons and
for those who provided no reason. We
note that the supporting and the contradicting reasons provided by students
varied widely in quality; however, an
exploratory analysis of the reasons
revealed no meaningful relationship.
In Table 4, we provide one-way
ANOVA results of our analysis of (a)
the relationship between students’ performance and asking them to provide
reasons that support their choice of
answers and (b) the relationship
between performance and asking students to provide reasons that contradict
their choice of answers (H5 and H6).
Because .0069 is less that .05, the null
hypothesis of equal means of performance level is rejected. In other words,
there was a statistically significant difference in students’ performance levels
between the group asked to provide reasons in support of its choices and the
group asked to contradict its choices.
The results reported in Tables 2 and 4
support our hypothesis that asking students to provide reasons supporting
their chosen answers causes them to
miss the chance to confront the possibility that they could be selecting the
wrong answer. Asking them to provide
reasons that contradict their answers not
only forces them to think critically
about their initial choices, but it also
gives them a second opportunity to
change their initially chosen answers if
the reason(s) for their initial selections
are found to be nonconvincing or less
logical.
Discussion
The findings of this study represent
the preliminary results of our attempt to
understand the effect of overconfidence
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
of business students on their performance on multiple-choice examinations.
Even though we used the multiplechoice format in this study, we believe
that the results could be generalized to
all types of examinations. We attempted to reduce overconfidence and
improve students’ performance on
multiple-choice examinations by asking
them to provide reasons contradicting or
supporting their chosen answers.
The results provide strong evidence
that high performing students appropriately exhibit more confidence in their
answers. High performing students are
also better calibrated and exhibit lower
levels of overconfidence. These students
are able to better discriminate between
what they know and what they do not
know. In contrast, although low performing students appropriately exhibit
lower confidence in their answers, they
are poorly calibrated. This suggests that
although high performing students are
more likely to recognize the extent and
limitation of their knowledge, low performing students have only limited
insight into their performance.
In general, students appropriately
exhibit greater confidence with easy
questions and lower confidence with
difficult questions. However, the students in this study seemed to be generally poorly calibrated with regard to difficult questions, with which they
demonstrated a significant level of overconfidence. This tendency could indicate that, on the whole, students do not
seem to appreciate the difficulty level of
the examination questions. One unexpected finding of the study is that the
students actually exhibited a slight
underconfidence with easy questions,
which suggests that they were uncertain
when encountering easy questions.
Thus, they may have attempted to read
more into the (easy) questions than was
actually warranted or necessary. Perhaps they thought that the easy questions were “trick” questions.
The results also show that asking
students to provide reasons that contradict their multiple-choice answers is
successful in reducing overconfidence
and improving student performance on
multiple-choice examinations. This
strategy seems to force them to study
more carefully the alternative answers
that they had initially rejected. Furthermore, it would seem that by concentrating on the weakness of their answers,
the students overtly recognize the limitations of their abilities, which results in
their becoming better calibrated and
thus exhibiting lower overconfidence.
In contrast, the results further show
that asking students to provide reasons
supporting
their
multiple-choice
answers did slightly increase overconfidence. However, we observed no statistically significant difference between
the group that provided reasons supporting their answers and the group that provided no reasons. This finding suggests
that students were already focusing on
reasons supporting their answers; as
such, asking them to provide supporting
reasons did not have a significant
impact.
Consistent with previous research in
psychology, we found that business students were overconfident when estimating the probability that their answers to
multiple-choice questions would be correct. The results of this study indicate
that instructors can remedy this false
sense of security that students demonstrate on multiple-choice examinations
by changing the format of multiplechoice examinations to require that the
student must not only provide the correct answer but also indicate why the
correct answer is correct or the incorrect
answer is incorrect. We contend that this
format not only will improve students’
performance but also will contribute
indirectly to learning, as it will help
make the student better calibrated and
thus less overconfident. A student who
is not overconfident will more likely
appreciate his or her lack of depth on
the subject matter and seek to remedy
the situation by studying.
Other equally interesting areas of
future research that could influence the
way that we evaluate students should
include the effect of question format
and presentation order on students’ confidence and performance on examinations. It would be interesting to see if
the order of presenting questions in
terms of difficulty (e.g., easy questions
followed by difficult ones) has any significant impact on students’ confidence
and thus their performance. An argument could be made that if students
encounter easy questions first, they
might become overconfident and their
exam performance might suffer. On the
other hand, if examination questions are
ordered from difficult to easy, students
might “panic” and the resulting performance might suffer also. Future
research on this issue could provide
invaluable insight for professors on how
best to evaluate students.
Furthermore, future research on other
methods of reducing overconfidence,
such as using computerized testing
methods that provide students with
immediate feedback on their performance on one question before they
move on to the next, might be interesting and worthwhile. Future research
could also focus on the effect of time
constraints on confidence, overconfidence, and performance.
The results of this study could have
significant implications, given the widespread use of the multiple-test format in
business education. The results indicate
that students indeed can be trained to
perform better on the standard multiplechoice exams—such as the Certified
Public Accountant (CPA) and Certified
Financial Analyst (CFA) exams—when
they are prepared through multiplechoice exams that ask them to provide
reasons contradicting their answer
selections.
REFERENCES
Arkes, H. R., Dawes, R. M., & Christensen, C.
(1986). Factors influencing the use of decision
rule in a probabilistic task. Organizational
Behavior and Human Decision Processes, 37,
93–110.
Arkes, H. R., Christensen, C., Lai, C., & Blumer,
C. (1987). Two methods of reducing overconfidence. Organizational Behavior and Human
Decision Processes, 39, 133–144.
Bar-Hillel, M., & Attali, Y. (2002). “The delicate
art of key balancing.” Or: When randomization
is too important to be trusted to chance. The
American Statistician, 564, 299–305.
Bobko, P., Roth, P. L., & Potosky, D. (1999).
Derivation and implications of a meta-analytic
matrix incorporating cognitive ability, alternative predictors, and job performance. Personnel
Psychology, 52, 561–589.
Hoch, S. J. (1985). Counterfactual reasoning and
accuracy in predicting personal events. Journal
of Experimental Psychology: Learning, Memory, and Cognition, 11, 719–731.
Jensen, A. R. (1985). The nature of Black-White
difference on various psychometric tests:
Spearman’s hypothesis. Behavioral and Brain
Sciences, 8, 193–263.
Koku, P. S., & Qureshi, A. A. (1999). Does confidence impact the performance on multiplechoice examinations? The case of marketing
March/April 2004
223
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
students. Academy of Marketing Science Congress, Malta.
Koriat, A., Lichtenstein, S., & Fischhoff, B.
(1980). Reasons for confidence. Journal of
Experimental Psychology: Human Learning
and Memory, 6, 107–118.
Lichtenstein, S., & Fischhoff, B. (1977). Do those
who know more also know more about how
much they know? Organizational Behavior and
Human Decision Processes, 20, 159–183.
Lichtenstein, S., Fischhoff, B., & Phillips, L. D.
224
Journal of Education for Business
(1982). Calibration of probabilities: The state
of the art to 1980. In D. Kahneman, P. Slovic, &
A. Tversky (Eds.), Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
Qureshi, A. A. (1998). Preliminary evidence about
the effect of confidence on exam performance.
American Academy of Accounting and
Finance, December.
Reynolds, C., & Brown, R. (1984). Perspectives
on bias in mental testing. New York: Plenum.
Sniezek, J., Paese, P., & Switzer, F., III. (1990).
The effect of choosing on confidence in choice.
Organizational Behavior and Human Decision
Processes, 46, 264–282.
Trafimow, D., & Sniezek, J. (1994). Perceived
performing and its effect on confidence. Organizational Behavior and Human Decision
Processes, 57, 290–302.
Wright, G. (1982). Changes in realism of probability assessments as a function of question
type. Acta Psychologica, 52, 165–174.