Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.79.5.275-282
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Students' Perceptions of Peer Evaluation: An
Expectancy Perspective
Yining Chen & Hao Lou
To cite this article: Yining Chen & Hao Lou (2004) Students' Perceptions of Peer Evaluation:
An Expectancy Perspective, Journal of Education for Business, 79:5, 275-282, DOI: 10.3200/
JOEB.79.5.275-282
To link to this article: http://dx.doi.org/10.3200/JOEB.79.5.275-282
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 90
View related articles
Citing articles: 17 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:26
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Students’ Perceptions of
Peer Evaluation:
An Expectancy Perspective
YINING CHEN
HAO LOU
Ohio University
Athens, Ohio
T
he use of student groups for class
projects is now a common pedagogical practice in many business
schools. Many instructors view class
group projects as excellent learning
exercises and training opportunities
(Beatty, Hass, & Sciglimpaglia, 1996;
Goldfinch & Raeside, 1990). Instructional methods have shifted from traditional
lecturing to collaborative learning1 based
on the principle that effective learning
requires students to be actively involved
in social learning contexts—that is,
group projects. Although most of the
commentary regarding the use of student
groups has been positive, one of the
instructional hazards of group projects is
evaluation (Gueldenzoph & May, 2002).
Because of the potential of uneven
effort and participation within groups,
evaluating a group member’s performance is often difficult. A method is
needed for evaluation and grading of
students with uneven workload distributions in project groups. This method
also must recognize the objectives, abilities, motivations, and contributions of
various individuals. Many researchers
believe that the problem of inequitable
contributions can be addressed effectively through a grading system that
gives appropriate weight to individual
contributions and to the group’s collective achievement (Williams, Beard, &
Rymer, 1991).
ABSTRACT. Because of the difficulty
of evaluating uneven performance
among group members, many researchers suggest incorporating peer evaluations in a grading system that permits
an instructor to evaluate and grade individual performance more equitably
within a group. In this study, the
authors employ expectancy theory to
assess key factors that may motivate
students to participate in the peer evaluation process of group projects.
Results show that students generally
thought that the most attractive outcome of peer evaluation is its use in the
determination of peers’ grades. The
second most attractive outcome is its
use for reduction of conflict and uneven
workload distribution among group
members. The authors provide implications for instructors planning to use
peer evaluation in group projects.
Previous research has suggested that
instructors potentially can use peer
assessment to improve evaluation accuracy. This research indicates that peer
assessment reflects the perspective of
peer students who are in close contact
and are familiar with group member
behaviors and characteristics that may
not otherwise be apparent to an instructor (Barrett, 1996; Cederblom & Lounsbury, 1980). Research on peer evaluations often examines issues such as the
development and validity of an evaluation instrument (Johnson & Smith, 1997;
Levi & Cadiz, 1998; Smith, 1998), the
validity (Beatty et al., 1996) and reliabil-
ity (Morahan-Martin, 1996) of peer ratings in measuring student performance,
and the potential bias of peer ratings
(Ghorpade & Lackritz, 2001; Hass,
Hass, & Wotruba, 1998). Very few studies, however, have examined students’
perceptions of peer evaluations and their
motivation to participate in the evaluation. Because students’ input is the root
and source of peer evaluation data, active
and meaningful participation by students
is essential. The peer evaluation data will
not be useful unless students willingly
provide quality input.
Expectancy theory has been recognized as one of the most promising conceptualizations of individual motivation
(Ferris, 1977). Many researchers have
proposed that expectancy theory provides an appropriate theoretical
research framework that examines a
user’s acceptance of and intent to use a
system (DeSanctis, 1983). In this study,
we use expectancy theory in a studentbased experiment that examines students’ acceptance of and motivation to
participate in a peer evaluation system.
We posed the following research questions: (a) Can expectancy theory appropriately explain the behavioral intention
of students to participate in peer evaluations? and (b) How do the potential uses
of peer evaluations affect students’
motivation to participate in the evaluation process?
May/June 2004
275
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Theoretical Background and
Supporting Literature
Group projects provide opportunities
for students to learn from each other
(Beatty et al., 1996). In a group setting,
students not only acquire the knowledge that other group members offer but
also acquire skills in group dynamics
and leadership that should prove valuable in their future business careers.
Previous research in this area points to
conclusive agreement as to the value of
group projects (Williams et al., 1991).
One primary problem surrounding the
implementation of group projects is
assigning each group member a fair
grade based on that member’s contribution to the group effort. Instructors need
a reward system for the inequitable
work done by students in project groups.
One solution is to institute a peer-rating
system that allows the instructor to
assign individual grades for a group
project, which, in turn, would encourage
full participation by all group members
(Cook, 1981, p. 50). The rationale
behind peer ratings is that individual
group members spend a substantial
amount of time working with each other
and, thus, are in a good position to recognize and assess their peers’ efforts and
contributions. Their insight can give
instructors a better understanding of the
dynamics that take place within individual groups (Hass et al., 1998, p. 200).
Functions of Peer Evaluation
In peer evaluation, group members
judge their fellow members on specific
traits, behavior, and achievements
(Kramer, 1990). Because peer ratings
involve the perspective of individuals of
the same level who are in close contact
with those being assessed (Barrett, 1996;
Cederblom & Lounsbury, 1980), peer
evaluations have been recognized widely
and adopted in both workplaces and
classrooms. Research indicates that peers
make finer distinctions among different
aspects of performance than do supervisors; consequently, feedback from peers
is more effective than supervisor ratings
in eliciting behavior changes (DeNisi,
Randolph, & Blencoe, 1982).
Peer evaluations traditionally have
served two functions in measuring
276
Journal of Education for Business
group members’ performances: a formative function and a summative one. At
the individual level, the formative function of peer evaluation provides information for individual group members to
modify behavior as necessary to assure
their grade is representative of their
effort. At the group level, the formative
function of peer evaluation facilitates
group productivity and group dynamics
(Gueldenzoph & May, 2002). It also
facilitates the resolution of differences
and conflict among group members
(Cook, 1981). The summative function
of peer evaluation provides information
for instructors in assigning individual
course grades. Peer rating supplies an
alternative source of information on
performance with a high degree of feedback specificity. For providing information on group members that is only
accessible by other group members,
composites of rating scores by student
peers and faculty members may yield a
more accurate assessment of performance (Sherrard & Raafat, 1994, p. 44).
Although peer evaluation shows
promise as a valid, reliable measure of
individual performance in a group setting, lack of participation or resistance
from students can limit its effectiveness
(Cederblom & Lounsbury, 1980).
Because students’ input is the basis of
peer evaluation data, students’ active participation and meaningful input are critical to the success of such a system. Very
few studies, however, have analyzed students’ attitudes toward peer evaluations
and the factors that influence their attitudes. Likewise, few studies have examined the behavioral intention of students’
participating in the peer evaluations.
(motivation) to participate in a system is
a strong predictor (and a more appropriate one than just attitudes) of the success of the system.
Expectancy theory is considered one
of the most promising conceptualizations of individual motivation. Originally developed by Vroom (1964), it has
served as a theoretical foundation for a
large body of studies in psychology,
organizational behavior, and management accounting (Brownell & McInnes,
1986; Geiger & Cooper, 1996; Hancock, 1995; Harrell, Caldwell, & Doty,
1985; Snead & Harrell, 1995).
Expectancy models are cognitive explanations of human behavior that cast a
person as an active, thinking, predicting
creature in his or her environment. He
or she continuously evaluates the outcomes of his or her behavior and subjectively assesses the likelihood that
each of his or her possible actions will
lead to various outcomes. The choice of
the amount of effort exerted by the individual is based on a systematic analysis
of (a) the values of the rewards from
these outcomes, (b) the likelihood that
rewards will result from these outcomes, and (c) the likelihood of reaching these outcomes through his or her
actions and effort.
According to Vroom, expectancy theory comprises two related models: the
valence model and the force model. In
our application of the theory, the valence
model shows that the overall attractiveness of a peer evaluation system to a student (V) is the summation of the products of the attractiveness of those
outcomes associated with the system
(Vk) and the probability that the system
will produce those outcomes (Ik):
Expectancy Theory
n
The theory of reasoned action, as proposed by Ajzen and Fishbein (1980), is
a well-researched model that has successfully predicted behavior in a variety
of contexts. Those researchers proposed
that attitudes and other variables (i.e., an
individual’s normative beliefs) do not
directly influence actual behavior (e.g.,
participation) but are fully mediated
through behavior intentions, or the
strength of intentions, to perform a specific behavior. This would imply that
measurement of behavioral intentions
V = ∑ (Vk ∗ I k )
k =1
where:
V = the valence, or attractiveness, of
a peer evaluation;
Vk = the valence, or attractiveness, of
peer evaluation outcome k; and
Ik = the perceived probability that the
peer evaluation will lead to outcome k.
In our case, the four potential outcomes (i.e., k = 1, 2, 3, 4) are the four
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
uses of peer evaluations described in the
literature (Cook, 1981; Gueldenzoph &
May, 2002; Sherrard & Raafat, 1994).
They are (a) determining peers’ grades,
(b) improving peers’ performance and
behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and uneven workload distribution among group members.
The second model, the force model,
shows that a student’s motivation to
exert effort in a peer evaluation system
(F) is the product of the attractiveness of
the system (V) and the probability that a
certain level of effort will result in a successful contribution to the system (E):
F=E*V
where:
F = the motivational force to participate in a peer evaluation;
E = the expectancy that a particular
level of participation, or effort, will
result in a successful contribution to the
evaluation; and
V = the valence, or attractiveness, of
the peer evaluation, derived in the previous equation of the valence model.
In summary, each student first uses
the valence model and then the force
model. In the valence model, each participant in a peer evaluation system
evaluates the system’s outcomes (e.g.,
determining peers’ grades, improving
peers’ performance and behavior,
enhancing group productivity and collaboration, and reducing conflict and
uneven workload distribution among
group members) and subjectively
assesses the likelihood that these outcomes will occur. Next, by placing his
or her own intrinsic values (or weights)
on the various outcomes, each student
evaluates the overall attractiveness of
the peer evaluation. Finally, the student
uses the force model to determine the
amount of effort that he or she is willing
to exert in the evaluation process. This
effort level is determined by the product
of the attractiveness generated by the
valence model (above) and the likelihood that his or her effort will result in
a successful contribution to the evaluation. Following this systematic analysis,
the student will determine how much
effort he or she would like to exert in
participating in the peer evaluation.
Research Objectives
The purpose of this study was
twofold. First, using valence model of
expectancy theory, we investigated the
impact of the potential uses (outcomes)
of peer evaluations on students’ perception of the attractiveness of participation
in the evaluation process. Specifically,
we tested four common uses of peer
evaluations that are well documented
and supported by literature. Of them,
three are formative and one is summative. The summative one is (a) determining peers’ grades; the three formative uses are (b) improving peers’
performance and behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and
uneven workload distribution among
group members. Second, using force
model of expectancy theory, we examined the influential factors of a student’s
motivation to participate in a peer evaluation. We specifically examined
whether a student’s motivation to participate is influenced largely by the perceived attractiveness of the peer evaluation or by his or her perception that his
or her actions will reach the expected
outcomes. We formulated two research
propositions based on the above
research objectives:
Proposition 1: The valence model
can explain a student’s perception of the
attractiveness of participation in a peer
evaluation.
Proposition 2: The force model can
explain a student’s motivation to participate in a peer evaluation.
Method
Subject Selection
In this study, we used MBA and upperlevel undergraduate business classes of
three medium-size (15,000 to 20,000
total enrollment) universities.2 All classes
used problem-based collaborative learning. The students were actively involved
in group projects and were familiar with
peer evaluation practices, so they were
considered appropriate subjects for this
study. We administered the instrument in
a regularly scheduled class session to all
the students who were present on that
particular day. We explained the use of
the instrument, read the instructions to
the students, and asked them to complete
the instrument. The entire process took
between 15 and 20 minutes. We obtained
122 usable instruments (see Table 1 for
the demographic information).
Experimental Design
The within-person or individual focus
of expectancy theory suggests that
appropriate tests of this theory should
involve comparing measurements of the
same individual’s motivation under different circumstances (Harrell et al.,
1985; Murky & Frizzier, 1986). In
response to this suggestion, this study
TABLE 1. Summary of Demographic Information
Gender: Male/female
Average GPA
Means
Collaborative or team-based project experiencea
Perception about peers (group members)b
Impression toward peer evaluation systemc
Level of comfort in evaluating other studentsd
Agreement with students evaluating each othere
67/55
3.39
2.60
2.51
0.60
2.39
2.82
a
The answer to the question “In general, how would you describe your collaborative or team-based
project experience from this institution?” was on a scale ranging from –5 (very bad) to 5 (very
good). bThe answer to the question “In general, how would you describe the peers (group members) you have had at this institution?” was on a scale from –5 (very bad) to 5 (very good). c The
answer to the question “What is your general impression about the peer evaluation system?” was
on a scale from –5 (useless) to 5 (very useful). d The answer to the question “How comfortable are
you in evaluating other students’ performance?” was on a scale ranging from –5 (very uncomfortable) to 5 (very comfortable). e The answer to the question “Do you agree with having students
evaluate each other in a collaborative or team-based project?” was on a scale ranging from –5
(strongly disagree) to 5 (strongly agree).
May/June 2004
277
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
incorporates a well-established withinperson methodology that was originally
developed by Stahl and Harrell (1981)
and was later proved valid by other
studies under various circumstances
(Geiger & Cooper, 1996; Snead & Harrell, 1995). This methodology uses a
judgment-modeling decision exercise
that provides a set of cues, which an
individual uses in arriving at a particular
judgment or decision. Multiple sets of
these cues are presented, each representing a unique combination of
strengths or values associated with the
cues. A separate judgment is required
from the individual for each unique
combination of cues presented.
We employed a one-half fractional
factorial design3 using the four outcomes of peer evaluations shown prior to
the making of Decision A. This resulted
in eight different combinations of the
outcomes (24 x 1/2 = 8 combinations).
We then presented each of the resulting
eight combinations at two levels (10%
and 90%) of expectancy to obtain 16
unique cases (8 combinations x 2 levels
of expectancy = 16 cases). This furnished each participant with multiple
cases that, in turn, provided multiple
measures of each individual’s behavioral
intentions under varied circumstances.4
This is a prerequisite for the withinperson application of expectancy theory
(Snead & Harrell, 1995).
In each of the 16 cases, we asked the
participants to make two decisions. The
first decision, Decision A, corresponded
to the V in the valence model and represented the overall attractiveness of participating in the peer evaluation, given
the likelihood (Ik = 10% or 90%) that the
four outcomes (Vk) would result from
their participation. (The instructions and
a sample case are provided in the Appendix). As mentioned earlier, the four outcomes are (a) determining peers’ grades,
(b) improving peers’ performance and
behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and uneven workload distribution among group members. The
second decision, Decision B, corresponded to F in the force model and
reflected the strength of a participant’s
motivation (or the level of effort a participant is willing to exert) to participate
in the peer evaluation, using (a) the
278
Journal of Education for Business
attractiveness of the evaluation (V)
obtained from Decision A and (b) the
expectancy (E = 10% or 90%) that if the
participant exerted a great deal of effort,
he or she would be successful in providing meaningful or useful input to the
evaluation process. We used an 11-point
response scale with a range of -5 to 5 for
Decision A and a range of 0 to 10 for
Decision B. For Decision A, -5 represented “very unattractive” and 5 represented “very attractive.” For Decision B,
0 represented “zero effort” and 10 represented a “great deal of effort.”5
We used “Further Information” in the
instrument to introduce the expectancy
(E) factor of the experiment. In peer
evaluations, it is quite common for
open-ended questions to be enclosed to
allow students to express an unconstrained opinion about a particular
aspect of a group member (peer) and/or
the group in which they are involved.
Such questions usually provide important diagnostic information and insight
for the formative evaluation about the
group member or the group. Although
important, open-ended questions are
more difficult to summarize and report.
In “Further Information,” we explained
to the participating student that despite
his or her best effort, his or her feedback
might not be helpful to the readers.
Likewise, the data from multiple-choice
questions could be difficult to interpret
or meaningless if the questionnaire is
designed poorly, the questions are
ambiguous, or the evaluation is administered inappropriately. Therefore, despite
making an effort, the student may not be
successful in contributing meaningfully
to the evaluation process. The students
were reminded further that their
participation in the evaluation was voluntary and that they were free to decide
to what extent they would participate.
This added the necessary uncertainty
(expectancy) about the reward of effort,
as well as providing a feeling that the
required effort could be considerable.
Results
Valence Model
Through the use of multiple regression analysis, we sought to determine
each student’s perception of the attrac-
tiveness of participating in the peer evaluation. Decision A (V) served as the
dependent variable, and the four secondlevel outcome instruments (Ik) served as
the independent variables. The resulting
standardized regression coefficients represented the relative importance (attractiveness) of each of the outcomes to each
participant in arriving at Decision A. In
Table 2, we present the mean adjusted R2
of the regressions and the mean standardized betas of each outcome. We do
not present detailed regression results
for each participant, but these are available from the authors.6
As indicated in Table 2, the mean R2
of the individual regression models was
.7103. The mean R2 represents the percentage of total variation in responses,
which is explained by the multiple
regression. The relatively high mean R2
indicates that the valence model of
expectancy theory explained much of
the variation in students’ perception of
the attractiveness of participating in a
peer evaluation and thus supports
Proposition 1.
The standardized betas of V1, V2,
V3, and V4 were all significant at the
level of .05 for more than half of the
students. This implies that all four of
the outcomes tested were important
factors to a majority of the students in
their determination of the attractiveness
of a peer evaluation system. Although
all four factors were important, some
factors were more important than others. It is the mean of these standardized
betas that explains how students, on
average, assess the attractiveness of
potential outcomes resulting from a
peer evaluation system. The participants, on average, placed the highest
valence on the outcome V1 and the second highest valance on the outcome V4.
The other outcomes, in descending
order of valence, were V3 and V2.
These results imply that peer evaluation
being used in determining peers’ grades
(V1) is the most attractive outcome to
students. Using peer evaluation in
reducing conflict and uneven workload
distribution among group members
(V4) is the second most attractive outcome. Students, however, consider the
use of peer evaluation results by students to enhance group productivity
and collaboration (V3) and to improve
peers’ performance and behavior (V2)
to be less important outcomes.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Force Model
We then used multiple regression
analysis to examine the force model
(Decision B) in the experiment. The
dependent variable is the individual’s
level of effort in participation in the peer
evaluation (F). The two independent
variables were (a) each student’s perception about the attractiveness of the peer
evaluation (V) from Decision A and (b)
the expectancy information (E = 10% or
90%), which is provided by the “Further
Information” sentence of the test instrument (see Appendix). We summarize the
force model results in Table 3.
The mean R2 (.7240) indicates that the
force model sufficiently explains the students’ motivation for participating in the
peer evaluation; therefore, it supports
Proposition 2. The mean standardized
regression coefficient F1 (.6136) indicates the impact of the overall attractiveness of the evaluation (V), whereas F2
(.4808) indicates the impact of the expectation that a certain level of effort leads to
successful contribution to the peer evaluation. Our results found a significant difference between the mean standardized
betas of F1 and F2. The p value of t test
was .0075. This result implies that the
attractiveness of the peer evaluation system (F1) was more important to the student’s motivation than the likelihood that
the student’s efforts would lead to successful contribution (F2). Students
apparently are willing to exert effort if
they find the peer evaluations (or outcomes of the peer evaluation) attractive
to them. Their motivation, however, is
less influenced by an inappropriately
designed evaluation, even though it hinders students from providing valid or
meaningful contribution and fails to
bring about the expected outcomes.
Internal Validity Tests
We used Pearson’s Correlations
between R2 values of valence and force
models and selected demographic information (gender, GPA, impression toward
collaborative learning, impression toward
peers, perception about the peer evaluation system, the level of comfort in eval-
uating peers, and the extent to which they
agree with students evaluating each
other) to test the associations between
the empirical results and participants’
background. We measured the demographic backgrounds, except for gender
and GPA, by 11-point scale questions.
The questions can be found in Table 1.
We asked the participants to evaluate
the 16 hypothetical cases (peer evaluations) presented to them instead of the
peer evaluations that they had experienced before. Therefore, the participants’ backgrounds should not have
affected their responses to these individual cases. Nonsignificant correlations
TABLE 2. Valence Model Regression Resultsa
Panel A. Regression statistics
Adjusted R 2
Standardized beta
weight
V1
V2
V3
V4
N
M
SD
Range
Frequency of
significance
at .05 level
122
.7103
.1429
.2266 to .9720
119/122 (98%)
122
122
122
122
.4347
.3234
.3588
.3624
.3523
.2063
.1907
.1883
–.8510 to .9826
.4440 to .7624
–.0917 to .8578
.2374 to .9019
99/122 (81%)
76/122 (62%)
78/122 (64%)
78/122 (64%)
Panel B. Ranking of outcomes and equality tests on standardized betas
Ranking
(from high to low)
Mean of
standardized betas
p value of t test
.4347
.3624
.3588
.3234
.0672 (V1 vs. V4)
.1260 (V4 vs. V3)
.1423 (V3 vs. V2)
V1
V4
V3
V2
a
Results (i.e., mean, standard deviation, range, and frequency of significant at .05) of individual
within-person regression models are reported in this table.
V1 = valence of determining peers’ grades.
V2 = valence of improving peers’ performances and behaviors.
V3 = valence of enhancing group productivity and collaboration.
V4 = valence of reducing conflict and uneven workload distribution among group members.
TABLE 3. Force Model Regression Resultsa
Adjusted R 2
Standardized beta
weight
F1
F2
n
M
SD
Range
Frequency of
significance
at .05 level
122
.7340
.1776
.0484 to .9823
119/122 (98%)
122
122
.6136
.4808
.2574
.3063
–.1333 to .9848
–.1487 to .9912
106/122 (87%)
81/122 (66%)
a
Results (i.e., mean, standard deviation, range, and frequency of significance at .05) of individual
within-person regression models are reported in this table.
F1 = weight placed on attractiveness of the peer evaluation.
F2 = weight placed on the expectancy of successfully participating in the peer evaluation.
May/June 2004
279
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
between participants’ background (i.e.,
gender, GPA, impression toward peers,
perception about the peer evaluation
system, etc.) and R2 values of valence
and force models would indicate that
the subjects are able to evaluate the proposed systems objectively without bias
and thus would support our argument
that the subjects that we used were
appropriate for this study.
The last two columns of Table 4 present Pearson’s Correlations between R2
values of valence and force models and
the participants’ demographic information. Of the 14 correlation coefficients,
only one was significant (at the .05 significance level). This result supports our
argument that the subjects were appropriate for this study because neither
their backgrounds nor their prior experiences with peer evaluations affected
their perceptions of the evaluation systems tested in the questionnaire.7
In Table 4, we also present correlations between demographic background
measures. An interesting finding was
that students who had more favorable
collaborative learning experiences had
more positive perceptions toward peers
and toward peer evaluations. This result
implies that students who consider peer
evaluation to be more useful and valu-
able generally have more positive comments about their peers and have more
favorable collaborative learning experience. In addition, students who considered peer evaluation to be more useful
and valuable were more comfortable in
evaluating other students’ performances, and more of those students
agreed that having students evaluate
each other in a collaborative learning
environment was a beneficial process.
Limitations and Conclusions
Limitations
Our study had some limitations. First,
the selection of subjects was not random. Students became subjects by virtue
of being present on the day that their
class was surveyed. The selection of
classes was arbitrary. Consequently,
caution should be used in the generalization of results to other groups and settings. Second, we used an experimental
task in this study, gathering the subjects’
responses in a controlled environment
rather than a real-world setting. Nevertheless, completing a peer evaluation in
a classroom and completing an instrument about peer evaluations in a classroom are similar activities. Third, stu-
dents were not given the opportunity for
input on the outcomes that motivated
them to participate in a peer evaluation.
In the instrument, four possible outcomes were given to the students. It is
possible that other possible outcomes of
peer evaluations may have had a
stronger impact on students’ motivations
than the four outcomes used. Future
research can solicit input from college
students on what specifically they see or
would like to see as the outcomes of an
evaluation system. Fourth, we used the
extreme levels of instrumentality and
expectancy (10% and 90%) in the cases.
This did not allow us to test for the full
range within the extremes. In another
sense, such extremes may not exist in
actual practice. Fifth, subjects came
from only one institution, which may
limit the applicability of the results to
other academic environments. Extensions can be made by future studies to
examine the effect of academic environments on the results of this study.
Concluding Remarks
The expectancy model used in this
study provided a good overall explanation of a student’s motivation to participate in the evaluation of peer effective-
TABLE 4. Pearson’s Correlation Coefficients (p values)
Gender
GPA
GPA
EXP
PER
IMP
COM
AGR
.0259
(.7890)
.0561
(.5392)
.0374
(.6892)
.1096
(.2294)
–.0338
(.7177)
.4801
(.0001)
–.0327
(.7205)
–.0414
(.6573)
.2759
(.0021)
.2272
(.0119)
.0556
(.5428)
–.0778
(.4046)
.0398
(.6638)
–.0068
(.9412)
.2913
(.0011)
–.1672
(.0656)
.0487
(.6024)
.0778
(.3947)
–.1204
(.1865)
.4147
(.0001)
.4606
(.0001)
EXP
PER
IMP
COM
AGR
GPA: grade point average.
EXP: collaborative or team-based project experience.
PER: perception about peers (group members).
IMP: impression toward peer evaluation system.
COM: level of comfort in evaluating other students’ performances.
AGR: agreement with students evaluating each other.
280
Journal of Education for Business
Adjusted R 2
force
Adjusted R 2
valence
–.0601
(.5110)
.0067
(.9426)
.1181
(.1951)
–.0128
(.8885)
.0449
(.6231)
.0558
(.5413)
–.0148
(.8714)
–.1497
(.0999)
.0977
(.2326)
.0898
(.3255)
–.0046
(.9604)
.0892
(.3287)
.1029
(.2250)
.2273
(.0118)
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
ness. The valence model significantly
explained student assessment of the
attractiveness of a peer evaluation system. Further, the force model provided a
good explanation of student motivation
to participate in the peer evaluation. By
the successful application of expectancy
theory, this study provides a better
understanding of the behavioral intention (motivation) of students’ participation in the peer evaluation process.
Our empirical results show that the
students have apparent preferences for
the uses of peer evaluations. Because
quality student participation is an essential antecedent of the success of peer
evaluations, this knowledge of student
motivation must be considered thoughtfully when the system is implemented.
However, if students are kept ignorant of
the use of peer evaluations, if peer evaluations are used for purposes that students
do not value, or if they see no visible
results from their participatory efforts,
they will cease to give meaningful input.
This research shows that students perceive determining peers’ grades as the
most attractive outcome for the use of
peer evaluation; reducing conflict and
uneven workload distribution was the
second most attractive outcome. Thus,
students who believe that their feedback
on peer evaluations will be considered in
determining peers’ grades and will result
in reduced conflict, uneven workload, or
both should be highly motivated to provide such feedback.
With the goal of motivating students
to participate in the peer evaluation
process, we make the following practical suggestions. First, consider listing
prominently the uses of the peer evaluation on the evaluation instrument. This
will inform the students of its uses. If
these uses are consistent with the uses
that students prefer (and they believe
that the evaluations will truly be used
for these purposes), the students will
assign a high valence to the evaluation
system. The next step is to show students that their feedback really will be
used. This will increase their tendency
to perceive that the expected outcomes
are likely on the evaluation. It would
also increase their tendency to perceive
that they will be successful in providing
meaningful feedback, because they see
that previous feedback has been used
successfully. Thus, their force or motivation to participate will be high. One
way of showing students that their feedback has been used successfully is to
encourage instructors to cite on the
course syllabus recent examples of how
peer evaluations have helped enhance
group productivity, improve peer performance and behavior, reduce conflict
and uneven workload, and determine
grades. This approach provides a lowcost but highly visible way to show students the benefits of peer evaluations. It
may also have the salutary effect of
encouraging faculty members to ponder
the information contained in student
evaluations and act on it.
NOTES
1. Collaborative learning refers to an instruction method in which students at various performance levels work together in small groups
toward a common goal. The students are responsible for one another’s learning as well as their own
(Gokhale, 1995).
2. We ran analyses for the MBA and upperlevel undergraduate business classes separately
and found no significant differences between the
two groups. Thus, we presented only combined
results in the results section.
3. According to Montgomery (1984, p. 325),
“if the experimenter can reasonably assume that
certain high-order interactions are negligible, then
information on main effects and low-order interactions may be obtained by running only a fraction of the complete factorial experiment.” A onehalf fraction of the 24 design can be found in
Montgomery (pp. 331–334). Prior expectancy theory studies (Burton, Chen, Grover, & Stewart,
1992; Snead & Harrell, 1995) also used one-half
fractional factorial design.
4. In a pilot test, we created two different
instruments; each had the order of the cases determined at random. We distributed the two instruments to every other student. We compared the
average R2s from the two random order versions
and found no significant differences between
them. This result implies that there is no order
effect in our experimental design.
5. The level of effort a student will apply in
completing the peer evaluation could indicate the
amount of thought given to responses, the desire to
provide meaningful written comments to the openended questions, and the willingness to complete
the evaluation form thoroughly. The expectancy
theory models an individual’s motivation. The
model uses each individual’s responses to measure
or assess how that individual values the possible
outcomes and how much effort the individual will
exert toward achieving those outcomes. Consequently, all students do not need to have the same
definition of what participation and effort mean.
6. We can be contacted at [email protected]
(Yining Chen) or [email protected] (Hao Lou).
7. It is reasonable to expect an association
between someone’s prior experience with an evaluation system and his or her motivation to participate
in that particular system. However, the participants
were asked to evaluate the 16 proposed cases (evaluation systems) but not the system that they experi-
enced. Therefore, the insignificant correlations
indicate that the subjects were able to evaluate the
proposed systems objectively without bias, thus
supporting our argument that the subjects whom we
used were appropriate for this study.
REFERENCES
Ajzen, I., & Fishbein, M. (1980). Understanding
attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall.
Barrett, R. S. (1996). Performance rating. Chicago: Science Research Associates.
Beatty, J. R., Haas, R. W., & Sciglimpaglia, D.
(1996). Using peer evaluations to assess individual performances in group class projects.
Journal of Marketing Education, 18(2), 17–27.
Brownell, P., & McInnes, M. (1986). Budgetary
participation, motivation, and managerial performance. Accounting Review, 61(4), 587–600.
Burton, F. G., Chen, Y., Grover, V., & Stewart, K.
A. (1992). An application of expectancy theory
for assessing user motivation to utilize an
expert system. Journal of Management Information Systems, 9(3), 183–198.
Cederblom, D., & Lounsbury, J. W. (1980). An
investigation of user acceptance of peer evaluations. Personnel Psychology, 33(3), 567–579.
Cook, R. W. (1981). An investigation of student
peer evaluation on group project performance.
Journal of Marketing Education, 3(1), 50–52.
DeNisi, A. S., Randolph, W. A., & Blencoe, A. G.
(1982). Level and source of feedback as determinants of feedback effectiveness. Proceedings
of the Academy of Management 42nd Annual
Meeting (pp. 175–179), New York.
DeSanctis, G. (1983). Expectancy theory as explanation of voluntary use of a decision support system. Psychological Reports, 52(1), 247–260.
Ferris, K. R. (1977). A test of the expectancy theory as motivation in an accounting environment. Accounting Review, 52(3), 605–614.
Geiger, M. A., & Cooper, E. A. (1996). Using
expectancy theory to assess student motivation.
Issues in Accounting Education, 11(1), 113–129.
Ghorpade, J., & Lackritz, J. R. (2001). Peer evaluation in the classroom: A check for sex and
race/ethnicity effects. Journal of Education for
Business, 76(5), 274–281.
Gokhale, A. A. (1995). Collaborative learning
enhances critical thinking. Journal of Technology Education, 7(1), 22–30.
Goldfinch, J., & Raeside, R. (1990). Development
of a peer assessment technique for obtaining
individual marks on a group project. Assessment and Evaluation in Higher Education,
15(3), 210–231.
Gueldenzoph, L. E., & May, G. L. (2002). Collaborative peer evaluation: Best practices for group
member assessments. Business Communication
Quarterly, 65(1), 9–20.
Haas, A. L., Hass, R. W., & Wotruba, T. R. (1998).
The use of self-ratings and peer ratings to evaluate performances of student group members.
Journal of Marketing Education, 20(3),
200–209.
Hancock, D. R. (1995). What teachers may do to
influence student motivation: An application of
expectancy theory. The Journal of General
Education, 44(3), 171–179.
Harrell, A. M., Caldwell, C., & Doty, E. (1985).
Within-person expectancy theory predictions of
accounting students’ motivation to achieve academic success. Accounting Review, 60(4),
724–735.
Johnson, C. B., & Smith, F. I. (1997). Assessment
of a complex peer evaluation instrument for
May/June 2004
281
team learning and group processes. Accounting
Education, 2(1), 21–40.
Kramer, J. F. (1990). Perceived similarity and
accuracy of peer ratings. Journal of Educational Psychology, 82(2), 213–218.
Levi, D., & Cadiz, D. (1998). Evaluating team
work on student projects: The use of behaviorally anchored scales to evaluate student performance. ERIC Document Reproduction Service, ED 424250.
Montgomery, D. C. (1984). Design and analysis
of experiments. New York: Wiley.
Morahan-Martin, J. (1996). Should peers’ evaluations be used in class projects? Questions
regarding reliability, leniency, and acceptance.
Psychological Reports, 78(3), 1243–1250.
Murky, D., & Frizzier, K. B. (1986). A within-subjects test of expectancy theory in a public
accounting environment. Journal of Accounting
Research, 24(2), 400–404.
Sherrard, W. R., & Raafat, F. (1994). An empirical
study of peer bias in evaluations: Students rating students. Journal of Education for Business,
70(1), 43–47.
Smith, K. A. (1998). Grading cooperative projects.
New Directions for Teaching and Learning,
74(2), 59–67.
Snead, K. C., & Harrell, A. M. (1995). An appli-
cation of expectancy theory to explain a manager’s intention to use a decision support system. Decision Sciences, 25(4), 499–513.
Stahl, M. J., & Harrell, A. M. (1981). Modeling
effort decisions with behavioral decision theory: Toward an individual differences model of
expectancy theory. Organizational Behavior
and Human Performance, 27(3), 303–325.
Vroom, V. C. (1964). Work and motivation. New
York: Wiley.
Williams, D. L., Beard, J. D., & Rymer, J.
(1991). Team projects: Achieving their full
potential. Journal of Marketing Education,
13(2), 45–53.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
APPENDIX
Instructions
As a student involved in collaborative or team-based projects, you are asked to evaluate the performance of your peers (group members). These peer evaluations may be
used in various ways, such as determining peers’ grades, improving peers’ performance and behavior, enhancing group productivity and collaboration, and reducing
conflict and uneven workload distribution.
This exercise presents 16 situations. Each situation is different with respect to how the
peer evaluation is likely to be used. We would like to know how attractive participation in such peer evaluation is to you in each given situation.
You are asked to make two decisions. You must first decide how attractive it would be
for you to participate in the peer evaluation (Decision A). Next you must decide how
much effort you would exert in completing the peer evaluation (Decision B). Use the
information provided in each situation to reach your decisions. There are no “right” or
“wrong” responses, so express your opinions freely. A sample situation is provided
below. The 16 different situations start on the next page.
EXAMPLE QUESTIONNAIRE
The likelihood that your peer evaluation feedback:
will be taken into consideration in determining
peers’ grades is …………………………...…………...……….…...…
will be provided to individual group member in
improving his or her performance and behavior is ...………..…...…
will be used to enhance productivity and collaboration
of the group in meeting its goal is ......…………...…...…...…...…...…
will be used to reduce conflict and uneven workload
distribution among group members is ..…...…...…...…...…...…...…...
HIGH (90%)
HIGH (90%)
HIGH (90%)
LOW (10%)
DECISION A: With the above outcomes and associated likelihood levels in mind,
indicate the attractiveness to you of participating in the peer evaluation.
–5
–4
–3
Very unattractive
–2
–1
0
+1
+2
+3
+4
+5
Very attractive
FURTHER INFORMATION: The peer evaluation contains several open-ended essay
questions, which will require a great deal of effort for you to complete. (As you know,
your participation in peer evaluations is voluntary. Thus you can choose to exert much
effort in the hopes of providing meaningful feedback or at the other extreme you can
do nothing.) If you exert a great deal of effort, the likelihood that the readers will find
your feedback helpful is ……………………………………………..... LOW (10%) *
DECISION B: Keeping in mind your attractiveness decision (DECISION A) and the
FURTHER INFORMATION, indicate the level of effort you would exert to participate in the peer evaluation.
0
1
Zero effort
2
3
4
5
6
7
8
9
10
Great deal of effort
* Despite your best effort to articulate your feelings, the peers may misinterpret your
feedback. Even the responses to multiple-choice questions are difficult to interpret
when the questions are designed poorly.
282
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Students' Perceptions of Peer Evaluation: An
Expectancy Perspective
Yining Chen & Hao Lou
To cite this article: Yining Chen & Hao Lou (2004) Students' Perceptions of Peer Evaluation:
An Expectancy Perspective, Journal of Education for Business, 79:5, 275-282, DOI: 10.3200/
JOEB.79.5.275-282
To link to this article: http://dx.doi.org/10.3200/JOEB.79.5.275-282
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 90
View related articles
Citing articles: 17 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:26
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Students’ Perceptions of
Peer Evaluation:
An Expectancy Perspective
YINING CHEN
HAO LOU
Ohio University
Athens, Ohio
T
he use of student groups for class
projects is now a common pedagogical practice in many business
schools. Many instructors view class
group projects as excellent learning
exercises and training opportunities
(Beatty, Hass, & Sciglimpaglia, 1996;
Goldfinch & Raeside, 1990). Instructional methods have shifted from traditional
lecturing to collaborative learning1 based
on the principle that effective learning
requires students to be actively involved
in social learning contexts—that is,
group projects. Although most of the
commentary regarding the use of student
groups has been positive, one of the
instructional hazards of group projects is
evaluation (Gueldenzoph & May, 2002).
Because of the potential of uneven
effort and participation within groups,
evaluating a group member’s performance is often difficult. A method is
needed for evaluation and grading of
students with uneven workload distributions in project groups. This method
also must recognize the objectives, abilities, motivations, and contributions of
various individuals. Many researchers
believe that the problem of inequitable
contributions can be addressed effectively through a grading system that
gives appropriate weight to individual
contributions and to the group’s collective achievement (Williams, Beard, &
Rymer, 1991).
ABSTRACT. Because of the difficulty
of evaluating uneven performance
among group members, many researchers suggest incorporating peer evaluations in a grading system that permits
an instructor to evaluate and grade individual performance more equitably
within a group. In this study, the
authors employ expectancy theory to
assess key factors that may motivate
students to participate in the peer evaluation process of group projects.
Results show that students generally
thought that the most attractive outcome of peer evaluation is its use in the
determination of peers’ grades. The
second most attractive outcome is its
use for reduction of conflict and uneven
workload distribution among group
members. The authors provide implications for instructors planning to use
peer evaluation in group projects.
Previous research has suggested that
instructors potentially can use peer
assessment to improve evaluation accuracy. This research indicates that peer
assessment reflects the perspective of
peer students who are in close contact
and are familiar with group member
behaviors and characteristics that may
not otherwise be apparent to an instructor (Barrett, 1996; Cederblom & Lounsbury, 1980). Research on peer evaluations often examines issues such as the
development and validity of an evaluation instrument (Johnson & Smith, 1997;
Levi & Cadiz, 1998; Smith, 1998), the
validity (Beatty et al., 1996) and reliabil-
ity (Morahan-Martin, 1996) of peer ratings in measuring student performance,
and the potential bias of peer ratings
(Ghorpade & Lackritz, 2001; Hass,
Hass, & Wotruba, 1998). Very few studies, however, have examined students’
perceptions of peer evaluations and their
motivation to participate in the evaluation. Because students’ input is the root
and source of peer evaluation data, active
and meaningful participation by students
is essential. The peer evaluation data will
not be useful unless students willingly
provide quality input.
Expectancy theory has been recognized as one of the most promising conceptualizations of individual motivation
(Ferris, 1977). Many researchers have
proposed that expectancy theory provides an appropriate theoretical
research framework that examines a
user’s acceptance of and intent to use a
system (DeSanctis, 1983). In this study,
we use expectancy theory in a studentbased experiment that examines students’ acceptance of and motivation to
participate in a peer evaluation system.
We posed the following research questions: (a) Can expectancy theory appropriately explain the behavioral intention
of students to participate in peer evaluations? and (b) How do the potential uses
of peer evaluations affect students’
motivation to participate in the evaluation process?
May/June 2004
275
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Theoretical Background and
Supporting Literature
Group projects provide opportunities
for students to learn from each other
(Beatty et al., 1996). In a group setting,
students not only acquire the knowledge that other group members offer but
also acquire skills in group dynamics
and leadership that should prove valuable in their future business careers.
Previous research in this area points to
conclusive agreement as to the value of
group projects (Williams et al., 1991).
One primary problem surrounding the
implementation of group projects is
assigning each group member a fair
grade based on that member’s contribution to the group effort. Instructors need
a reward system for the inequitable
work done by students in project groups.
One solution is to institute a peer-rating
system that allows the instructor to
assign individual grades for a group
project, which, in turn, would encourage
full participation by all group members
(Cook, 1981, p. 50). The rationale
behind peer ratings is that individual
group members spend a substantial
amount of time working with each other
and, thus, are in a good position to recognize and assess their peers’ efforts and
contributions. Their insight can give
instructors a better understanding of the
dynamics that take place within individual groups (Hass et al., 1998, p. 200).
Functions of Peer Evaluation
In peer evaluation, group members
judge their fellow members on specific
traits, behavior, and achievements
(Kramer, 1990). Because peer ratings
involve the perspective of individuals of
the same level who are in close contact
with those being assessed (Barrett, 1996;
Cederblom & Lounsbury, 1980), peer
evaluations have been recognized widely
and adopted in both workplaces and
classrooms. Research indicates that peers
make finer distinctions among different
aspects of performance than do supervisors; consequently, feedback from peers
is more effective than supervisor ratings
in eliciting behavior changes (DeNisi,
Randolph, & Blencoe, 1982).
Peer evaluations traditionally have
served two functions in measuring
276
Journal of Education for Business
group members’ performances: a formative function and a summative one. At
the individual level, the formative function of peer evaluation provides information for individual group members to
modify behavior as necessary to assure
their grade is representative of their
effort. At the group level, the formative
function of peer evaluation facilitates
group productivity and group dynamics
(Gueldenzoph & May, 2002). It also
facilitates the resolution of differences
and conflict among group members
(Cook, 1981). The summative function
of peer evaluation provides information
for instructors in assigning individual
course grades. Peer rating supplies an
alternative source of information on
performance with a high degree of feedback specificity. For providing information on group members that is only
accessible by other group members,
composites of rating scores by student
peers and faculty members may yield a
more accurate assessment of performance (Sherrard & Raafat, 1994, p. 44).
Although peer evaluation shows
promise as a valid, reliable measure of
individual performance in a group setting, lack of participation or resistance
from students can limit its effectiveness
(Cederblom & Lounsbury, 1980).
Because students’ input is the basis of
peer evaluation data, students’ active participation and meaningful input are critical to the success of such a system. Very
few studies, however, have analyzed students’ attitudes toward peer evaluations
and the factors that influence their attitudes. Likewise, few studies have examined the behavioral intention of students’
participating in the peer evaluations.
(motivation) to participate in a system is
a strong predictor (and a more appropriate one than just attitudes) of the success of the system.
Expectancy theory is considered one
of the most promising conceptualizations of individual motivation. Originally developed by Vroom (1964), it has
served as a theoretical foundation for a
large body of studies in psychology,
organizational behavior, and management accounting (Brownell & McInnes,
1986; Geiger & Cooper, 1996; Hancock, 1995; Harrell, Caldwell, & Doty,
1985; Snead & Harrell, 1995).
Expectancy models are cognitive explanations of human behavior that cast a
person as an active, thinking, predicting
creature in his or her environment. He
or she continuously evaluates the outcomes of his or her behavior and subjectively assesses the likelihood that
each of his or her possible actions will
lead to various outcomes. The choice of
the amount of effort exerted by the individual is based on a systematic analysis
of (a) the values of the rewards from
these outcomes, (b) the likelihood that
rewards will result from these outcomes, and (c) the likelihood of reaching these outcomes through his or her
actions and effort.
According to Vroom, expectancy theory comprises two related models: the
valence model and the force model. In
our application of the theory, the valence
model shows that the overall attractiveness of a peer evaluation system to a student (V) is the summation of the products of the attractiveness of those
outcomes associated with the system
(Vk) and the probability that the system
will produce those outcomes (Ik):
Expectancy Theory
n
The theory of reasoned action, as proposed by Ajzen and Fishbein (1980), is
a well-researched model that has successfully predicted behavior in a variety
of contexts. Those researchers proposed
that attitudes and other variables (i.e., an
individual’s normative beliefs) do not
directly influence actual behavior (e.g.,
participation) but are fully mediated
through behavior intentions, or the
strength of intentions, to perform a specific behavior. This would imply that
measurement of behavioral intentions
V = ∑ (Vk ∗ I k )
k =1
where:
V = the valence, or attractiveness, of
a peer evaluation;
Vk = the valence, or attractiveness, of
peer evaluation outcome k; and
Ik = the perceived probability that the
peer evaluation will lead to outcome k.
In our case, the four potential outcomes (i.e., k = 1, 2, 3, 4) are the four
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
uses of peer evaluations described in the
literature (Cook, 1981; Gueldenzoph &
May, 2002; Sherrard & Raafat, 1994).
They are (a) determining peers’ grades,
(b) improving peers’ performance and
behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and uneven workload distribution among group members.
The second model, the force model,
shows that a student’s motivation to
exert effort in a peer evaluation system
(F) is the product of the attractiveness of
the system (V) and the probability that a
certain level of effort will result in a successful contribution to the system (E):
F=E*V
where:
F = the motivational force to participate in a peer evaluation;
E = the expectancy that a particular
level of participation, or effort, will
result in a successful contribution to the
evaluation; and
V = the valence, or attractiveness, of
the peer evaluation, derived in the previous equation of the valence model.
In summary, each student first uses
the valence model and then the force
model. In the valence model, each participant in a peer evaluation system
evaluates the system’s outcomes (e.g.,
determining peers’ grades, improving
peers’ performance and behavior,
enhancing group productivity and collaboration, and reducing conflict and
uneven workload distribution among
group members) and subjectively
assesses the likelihood that these outcomes will occur. Next, by placing his
or her own intrinsic values (or weights)
on the various outcomes, each student
evaluates the overall attractiveness of
the peer evaluation. Finally, the student
uses the force model to determine the
amount of effort that he or she is willing
to exert in the evaluation process. This
effort level is determined by the product
of the attractiveness generated by the
valence model (above) and the likelihood that his or her effort will result in
a successful contribution to the evaluation. Following this systematic analysis,
the student will determine how much
effort he or she would like to exert in
participating in the peer evaluation.
Research Objectives
The purpose of this study was
twofold. First, using valence model of
expectancy theory, we investigated the
impact of the potential uses (outcomes)
of peer evaluations on students’ perception of the attractiveness of participation
in the evaluation process. Specifically,
we tested four common uses of peer
evaluations that are well documented
and supported by literature. Of them,
three are formative and one is summative. The summative one is (a) determining peers’ grades; the three formative uses are (b) improving peers’
performance and behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and
uneven workload distribution among
group members. Second, using force
model of expectancy theory, we examined the influential factors of a student’s
motivation to participate in a peer evaluation. We specifically examined
whether a student’s motivation to participate is influenced largely by the perceived attractiveness of the peer evaluation or by his or her perception that his
or her actions will reach the expected
outcomes. We formulated two research
propositions based on the above
research objectives:
Proposition 1: The valence model
can explain a student’s perception of the
attractiveness of participation in a peer
evaluation.
Proposition 2: The force model can
explain a student’s motivation to participate in a peer evaluation.
Method
Subject Selection
In this study, we used MBA and upperlevel undergraduate business classes of
three medium-size (15,000 to 20,000
total enrollment) universities.2 All classes
used problem-based collaborative learning. The students were actively involved
in group projects and were familiar with
peer evaluation practices, so they were
considered appropriate subjects for this
study. We administered the instrument in
a regularly scheduled class session to all
the students who were present on that
particular day. We explained the use of
the instrument, read the instructions to
the students, and asked them to complete
the instrument. The entire process took
between 15 and 20 minutes. We obtained
122 usable instruments (see Table 1 for
the demographic information).
Experimental Design
The within-person or individual focus
of expectancy theory suggests that
appropriate tests of this theory should
involve comparing measurements of the
same individual’s motivation under different circumstances (Harrell et al.,
1985; Murky & Frizzier, 1986). In
response to this suggestion, this study
TABLE 1. Summary of Demographic Information
Gender: Male/female
Average GPA
Means
Collaborative or team-based project experiencea
Perception about peers (group members)b
Impression toward peer evaluation systemc
Level of comfort in evaluating other studentsd
Agreement with students evaluating each othere
67/55
3.39
2.60
2.51
0.60
2.39
2.82
a
The answer to the question “In general, how would you describe your collaborative or team-based
project experience from this institution?” was on a scale ranging from –5 (very bad) to 5 (very
good). bThe answer to the question “In general, how would you describe the peers (group members) you have had at this institution?” was on a scale from –5 (very bad) to 5 (very good). c The
answer to the question “What is your general impression about the peer evaluation system?” was
on a scale from –5 (useless) to 5 (very useful). d The answer to the question “How comfortable are
you in evaluating other students’ performance?” was on a scale ranging from –5 (very uncomfortable) to 5 (very comfortable). e The answer to the question “Do you agree with having students
evaluate each other in a collaborative or team-based project?” was on a scale ranging from –5
(strongly disagree) to 5 (strongly agree).
May/June 2004
277
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
incorporates a well-established withinperson methodology that was originally
developed by Stahl and Harrell (1981)
and was later proved valid by other
studies under various circumstances
(Geiger & Cooper, 1996; Snead & Harrell, 1995). This methodology uses a
judgment-modeling decision exercise
that provides a set of cues, which an
individual uses in arriving at a particular
judgment or decision. Multiple sets of
these cues are presented, each representing a unique combination of
strengths or values associated with the
cues. A separate judgment is required
from the individual for each unique
combination of cues presented.
We employed a one-half fractional
factorial design3 using the four outcomes of peer evaluations shown prior to
the making of Decision A. This resulted
in eight different combinations of the
outcomes (24 x 1/2 = 8 combinations).
We then presented each of the resulting
eight combinations at two levels (10%
and 90%) of expectancy to obtain 16
unique cases (8 combinations x 2 levels
of expectancy = 16 cases). This furnished each participant with multiple
cases that, in turn, provided multiple
measures of each individual’s behavioral
intentions under varied circumstances.4
This is a prerequisite for the withinperson application of expectancy theory
(Snead & Harrell, 1995).
In each of the 16 cases, we asked the
participants to make two decisions. The
first decision, Decision A, corresponded
to the V in the valence model and represented the overall attractiveness of participating in the peer evaluation, given
the likelihood (Ik = 10% or 90%) that the
four outcomes (Vk) would result from
their participation. (The instructions and
a sample case are provided in the Appendix). As mentioned earlier, the four outcomes are (a) determining peers’ grades,
(b) improving peers’ performance and
behavior, (c) enhancing group productivity and collaboration, and (d) reducing conflict and uneven workload distribution among group members. The
second decision, Decision B, corresponded to F in the force model and
reflected the strength of a participant’s
motivation (or the level of effort a participant is willing to exert) to participate
in the peer evaluation, using (a) the
278
Journal of Education for Business
attractiveness of the evaluation (V)
obtained from Decision A and (b) the
expectancy (E = 10% or 90%) that if the
participant exerted a great deal of effort,
he or she would be successful in providing meaningful or useful input to the
evaluation process. We used an 11-point
response scale with a range of -5 to 5 for
Decision A and a range of 0 to 10 for
Decision B. For Decision A, -5 represented “very unattractive” and 5 represented “very attractive.” For Decision B,
0 represented “zero effort” and 10 represented a “great deal of effort.”5
We used “Further Information” in the
instrument to introduce the expectancy
(E) factor of the experiment. In peer
evaluations, it is quite common for
open-ended questions to be enclosed to
allow students to express an unconstrained opinion about a particular
aspect of a group member (peer) and/or
the group in which they are involved.
Such questions usually provide important diagnostic information and insight
for the formative evaluation about the
group member or the group. Although
important, open-ended questions are
more difficult to summarize and report.
In “Further Information,” we explained
to the participating student that despite
his or her best effort, his or her feedback
might not be helpful to the readers.
Likewise, the data from multiple-choice
questions could be difficult to interpret
or meaningless if the questionnaire is
designed poorly, the questions are
ambiguous, or the evaluation is administered inappropriately. Therefore, despite
making an effort, the student may not be
successful in contributing meaningfully
to the evaluation process. The students
were reminded further that their
participation in the evaluation was voluntary and that they were free to decide
to what extent they would participate.
This added the necessary uncertainty
(expectancy) about the reward of effort,
as well as providing a feeling that the
required effort could be considerable.
Results
Valence Model
Through the use of multiple regression analysis, we sought to determine
each student’s perception of the attrac-
tiveness of participating in the peer evaluation. Decision A (V) served as the
dependent variable, and the four secondlevel outcome instruments (Ik) served as
the independent variables. The resulting
standardized regression coefficients represented the relative importance (attractiveness) of each of the outcomes to each
participant in arriving at Decision A. In
Table 2, we present the mean adjusted R2
of the regressions and the mean standardized betas of each outcome. We do
not present detailed regression results
for each participant, but these are available from the authors.6
As indicated in Table 2, the mean R2
of the individual regression models was
.7103. The mean R2 represents the percentage of total variation in responses,
which is explained by the multiple
regression. The relatively high mean R2
indicates that the valence model of
expectancy theory explained much of
the variation in students’ perception of
the attractiveness of participating in a
peer evaluation and thus supports
Proposition 1.
The standardized betas of V1, V2,
V3, and V4 were all significant at the
level of .05 for more than half of the
students. This implies that all four of
the outcomes tested were important
factors to a majority of the students in
their determination of the attractiveness
of a peer evaluation system. Although
all four factors were important, some
factors were more important than others. It is the mean of these standardized
betas that explains how students, on
average, assess the attractiveness of
potential outcomes resulting from a
peer evaluation system. The participants, on average, placed the highest
valence on the outcome V1 and the second highest valance on the outcome V4.
The other outcomes, in descending
order of valence, were V3 and V2.
These results imply that peer evaluation
being used in determining peers’ grades
(V1) is the most attractive outcome to
students. Using peer evaluation in
reducing conflict and uneven workload
distribution among group members
(V4) is the second most attractive outcome. Students, however, consider the
use of peer evaluation results by students to enhance group productivity
and collaboration (V3) and to improve
peers’ performance and behavior (V2)
to be less important outcomes.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
Force Model
We then used multiple regression
analysis to examine the force model
(Decision B) in the experiment. The
dependent variable is the individual’s
level of effort in participation in the peer
evaluation (F). The two independent
variables were (a) each student’s perception about the attractiveness of the peer
evaluation (V) from Decision A and (b)
the expectancy information (E = 10% or
90%), which is provided by the “Further
Information” sentence of the test instrument (see Appendix). We summarize the
force model results in Table 3.
The mean R2 (.7240) indicates that the
force model sufficiently explains the students’ motivation for participating in the
peer evaluation; therefore, it supports
Proposition 2. The mean standardized
regression coefficient F1 (.6136) indicates the impact of the overall attractiveness of the evaluation (V), whereas F2
(.4808) indicates the impact of the expectation that a certain level of effort leads to
successful contribution to the peer evaluation. Our results found a significant difference between the mean standardized
betas of F1 and F2. The p value of t test
was .0075. This result implies that the
attractiveness of the peer evaluation system (F1) was more important to the student’s motivation than the likelihood that
the student’s efforts would lead to successful contribution (F2). Students
apparently are willing to exert effort if
they find the peer evaluations (or outcomes of the peer evaluation) attractive
to them. Their motivation, however, is
less influenced by an inappropriately
designed evaluation, even though it hinders students from providing valid or
meaningful contribution and fails to
bring about the expected outcomes.
Internal Validity Tests
We used Pearson’s Correlations
between R2 values of valence and force
models and selected demographic information (gender, GPA, impression toward
collaborative learning, impression toward
peers, perception about the peer evaluation system, the level of comfort in eval-
uating peers, and the extent to which they
agree with students evaluating each
other) to test the associations between
the empirical results and participants’
background. We measured the demographic backgrounds, except for gender
and GPA, by 11-point scale questions.
The questions can be found in Table 1.
We asked the participants to evaluate
the 16 hypothetical cases (peer evaluations) presented to them instead of the
peer evaluations that they had experienced before. Therefore, the participants’ backgrounds should not have
affected their responses to these individual cases. Nonsignificant correlations
TABLE 2. Valence Model Regression Resultsa
Panel A. Regression statistics
Adjusted R 2
Standardized beta
weight
V1
V2
V3
V4
N
M
SD
Range
Frequency of
significance
at .05 level
122
.7103
.1429
.2266 to .9720
119/122 (98%)
122
122
122
122
.4347
.3234
.3588
.3624
.3523
.2063
.1907
.1883
–.8510 to .9826
.4440 to .7624
–.0917 to .8578
.2374 to .9019
99/122 (81%)
76/122 (62%)
78/122 (64%)
78/122 (64%)
Panel B. Ranking of outcomes and equality tests on standardized betas
Ranking
(from high to low)
Mean of
standardized betas
p value of t test
.4347
.3624
.3588
.3234
.0672 (V1 vs. V4)
.1260 (V4 vs. V3)
.1423 (V3 vs. V2)
V1
V4
V3
V2
a
Results (i.e., mean, standard deviation, range, and frequency of significant at .05) of individual
within-person regression models are reported in this table.
V1 = valence of determining peers’ grades.
V2 = valence of improving peers’ performances and behaviors.
V3 = valence of enhancing group productivity and collaboration.
V4 = valence of reducing conflict and uneven workload distribution among group members.
TABLE 3. Force Model Regression Resultsa
Adjusted R 2
Standardized beta
weight
F1
F2
n
M
SD
Range
Frequency of
significance
at .05 level
122
.7340
.1776
.0484 to .9823
119/122 (98%)
122
122
.6136
.4808
.2574
.3063
–.1333 to .9848
–.1487 to .9912
106/122 (87%)
81/122 (66%)
a
Results (i.e., mean, standard deviation, range, and frequency of significance at .05) of individual
within-person regression models are reported in this table.
F1 = weight placed on attractiveness of the peer evaluation.
F2 = weight placed on the expectancy of successfully participating in the peer evaluation.
May/June 2004
279
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
between participants’ background (i.e.,
gender, GPA, impression toward peers,
perception about the peer evaluation
system, etc.) and R2 values of valence
and force models would indicate that
the subjects are able to evaluate the proposed systems objectively without bias
and thus would support our argument
that the subjects that we used were
appropriate for this study.
The last two columns of Table 4 present Pearson’s Correlations between R2
values of valence and force models and
the participants’ demographic information. Of the 14 correlation coefficients,
only one was significant (at the .05 significance level). This result supports our
argument that the subjects were appropriate for this study because neither
their backgrounds nor their prior experiences with peer evaluations affected
their perceptions of the evaluation systems tested in the questionnaire.7
In Table 4, we also present correlations between demographic background
measures. An interesting finding was
that students who had more favorable
collaborative learning experiences had
more positive perceptions toward peers
and toward peer evaluations. This result
implies that students who consider peer
evaluation to be more useful and valu-
able generally have more positive comments about their peers and have more
favorable collaborative learning experience. In addition, students who considered peer evaluation to be more useful
and valuable were more comfortable in
evaluating other students’ performances, and more of those students
agreed that having students evaluate
each other in a collaborative learning
environment was a beneficial process.
Limitations and Conclusions
Limitations
Our study had some limitations. First,
the selection of subjects was not random. Students became subjects by virtue
of being present on the day that their
class was surveyed. The selection of
classes was arbitrary. Consequently,
caution should be used in the generalization of results to other groups and settings. Second, we used an experimental
task in this study, gathering the subjects’
responses in a controlled environment
rather than a real-world setting. Nevertheless, completing a peer evaluation in
a classroom and completing an instrument about peer evaluations in a classroom are similar activities. Third, stu-
dents were not given the opportunity for
input on the outcomes that motivated
them to participate in a peer evaluation.
In the instrument, four possible outcomes were given to the students. It is
possible that other possible outcomes of
peer evaluations may have had a
stronger impact on students’ motivations
than the four outcomes used. Future
research can solicit input from college
students on what specifically they see or
would like to see as the outcomes of an
evaluation system. Fourth, we used the
extreme levels of instrumentality and
expectancy (10% and 90%) in the cases.
This did not allow us to test for the full
range within the extremes. In another
sense, such extremes may not exist in
actual practice. Fifth, subjects came
from only one institution, which may
limit the applicability of the results to
other academic environments. Extensions can be made by future studies to
examine the effect of academic environments on the results of this study.
Concluding Remarks
The expectancy model used in this
study provided a good overall explanation of a student’s motivation to participate in the evaluation of peer effective-
TABLE 4. Pearson’s Correlation Coefficients (p values)
Gender
GPA
GPA
EXP
PER
IMP
COM
AGR
.0259
(.7890)
.0561
(.5392)
.0374
(.6892)
.1096
(.2294)
–.0338
(.7177)
.4801
(.0001)
–.0327
(.7205)
–.0414
(.6573)
.2759
(.0021)
.2272
(.0119)
.0556
(.5428)
–.0778
(.4046)
.0398
(.6638)
–.0068
(.9412)
.2913
(.0011)
–.1672
(.0656)
.0487
(.6024)
.0778
(.3947)
–.1204
(.1865)
.4147
(.0001)
.4606
(.0001)
EXP
PER
IMP
COM
AGR
GPA: grade point average.
EXP: collaborative or team-based project experience.
PER: perception about peers (group members).
IMP: impression toward peer evaluation system.
COM: level of comfort in evaluating other students’ performances.
AGR: agreement with students evaluating each other.
280
Journal of Education for Business
Adjusted R 2
force
Adjusted R 2
valence
–.0601
(.5110)
.0067
(.9426)
.1181
(.1951)
–.0128
(.8885)
.0449
(.6231)
.0558
(.5413)
–.0148
(.8714)
–.1497
(.0999)
.0977
(.2326)
.0898
(.3255)
–.0046
(.9604)
.0892
(.3287)
.1029
(.2250)
.2273
(.0118)
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
ness. The valence model significantly
explained student assessment of the
attractiveness of a peer evaluation system. Further, the force model provided a
good explanation of student motivation
to participate in the peer evaluation. By
the successful application of expectancy
theory, this study provides a better
understanding of the behavioral intention (motivation) of students’ participation in the peer evaluation process.
Our empirical results show that the
students have apparent preferences for
the uses of peer evaluations. Because
quality student participation is an essential antecedent of the success of peer
evaluations, this knowledge of student
motivation must be considered thoughtfully when the system is implemented.
However, if students are kept ignorant of
the use of peer evaluations, if peer evaluations are used for purposes that students
do not value, or if they see no visible
results from their participatory efforts,
they will cease to give meaningful input.
This research shows that students perceive determining peers’ grades as the
most attractive outcome for the use of
peer evaluation; reducing conflict and
uneven workload distribution was the
second most attractive outcome. Thus,
students who believe that their feedback
on peer evaluations will be considered in
determining peers’ grades and will result
in reduced conflict, uneven workload, or
both should be highly motivated to provide such feedback.
With the goal of motivating students
to participate in the peer evaluation
process, we make the following practical suggestions. First, consider listing
prominently the uses of the peer evaluation on the evaluation instrument. This
will inform the students of its uses. If
these uses are consistent with the uses
that students prefer (and they believe
that the evaluations will truly be used
for these purposes), the students will
assign a high valence to the evaluation
system. The next step is to show students that their feedback really will be
used. This will increase their tendency
to perceive that the expected outcomes
are likely on the evaluation. It would
also increase their tendency to perceive
that they will be successful in providing
meaningful feedback, because they see
that previous feedback has been used
successfully. Thus, their force or motivation to participate will be high. One
way of showing students that their feedback has been used successfully is to
encourage instructors to cite on the
course syllabus recent examples of how
peer evaluations have helped enhance
group productivity, improve peer performance and behavior, reduce conflict
and uneven workload, and determine
grades. This approach provides a lowcost but highly visible way to show students the benefits of peer evaluations. It
may also have the salutary effect of
encouraging faculty members to ponder
the information contained in student
evaluations and act on it.
NOTES
1. Collaborative learning refers to an instruction method in which students at various performance levels work together in small groups
toward a common goal. The students are responsible for one another’s learning as well as their own
(Gokhale, 1995).
2. We ran analyses for the MBA and upperlevel undergraduate business classes separately
and found no significant differences between the
two groups. Thus, we presented only combined
results in the results section.
3. According to Montgomery (1984, p. 325),
“if the experimenter can reasonably assume that
certain high-order interactions are negligible, then
information on main effects and low-order interactions may be obtained by running only a fraction of the complete factorial experiment.” A onehalf fraction of the 24 design can be found in
Montgomery (pp. 331–334). Prior expectancy theory studies (Burton, Chen, Grover, & Stewart,
1992; Snead & Harrell, 1995) also used one-half
fractional factorial design.
4. In a pilot test, we created two different
instruments; each had the order of the cases determined at random. We distributed the two instruments to every other student. We compared the
average R2s from the two random order versions
and found no significant differences between
them. This result implies that there is no order
effect in our experimental design.
5. The level of effort a student will apply in
completing the peer evaluation could indicate the
amount of thought given to responses, the desire to
provide meaningful written comments to the openended questions, and the willingness to complete
the evaluation form thoroughly. The expectancy
theory models an individual’s motivation. The
model uses each individual’s responses to measure
or assess how that individual values the possible
outcomes and how much effort the individual will
exert toward achieving those outcomes. Consequently, all students do not need to have the same
definition of what participation and effort mean.
6. We can be contacted at [email protected]
(Yining Chen) or [email protected] (Hao Lou).
7. It is reasonable to expect an association
between someone’s prior experience with an evaluation system and his or her motivation to participate
in that particular system. However, the participants
were asked to evaluate the 16 proposed cases (evaluation systems) but not the system that they experi-
enced. Therefore, the insignificant correlations
indicate that the subjects were able to evaluate the
proposed systems objectively without bias, thus
supporting our argument that the subjects whom we
used were appropriate for this study.
REFERENCES
Ajzen, I., & Fishbein, M. (1980). Understanding
attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall.
Barrett, R. S. (1996). Performance rating. Chicago: Science Research Associates.
Beatty, J. R., Haas, R. W., & Sciglimpaglia, D.
(1996). Using peer evaluations to assess individual performances in group class projects.
Journal of Marketing Education, 18(2), 17–27.
Brownell, P., & McInnes, M. (1986). Budgetary
participation, motivation, and managerial performance. Accounting Review, 61(4), 587–600.
Burton, F. G., Chen, Y., Grover, V., & Stewart, K.
A. (1992). An application of expectancy theory
for assessing user motivation to utilize an
expert system. Journal of Management Information Systems, 9(3), 183–198.
Cederblom, D., & Lounsbury, J. W. (1980). An
investigation of user acceptance of peer evaluations. Personnel Psychology, 33(3), 567–579.
Cook, R. W. (1981). An investigation of student
peer evaluation on group project performance.
Journal of Marketing Education, 3(1), 50–52.
DeNisi, A. S., Randolph, W. A., & Blencoe, A. G.
(1982). Level and source of feedback as determinants of feedback effectiveness. Proceedings
of the Academy of Management 42nd Annual
Meeting (pp. 175–179), New York.
DeSanctis, G. (1983). Expectancy theory as explanation of voluntary use of a decision support system. Psychological Reports, 52(1), 247–260.
Ferris, K. R. (1977). A test of the expectancy theory as motivation in an accounting environment. Accounting Review, 52(3), 605–614.
Geiger, M. A., & Cooper, E. A. (1996). Using
expectancy theory to assess student motivation.
Issues in Accounting Education, 11(1), 113–129.
Ghorpade, J., & Lackritz, J. R. (2001). Peer evaluation in the classroom: A check for sex and
race/ethnicity effects. Journal of Education for
Business, 76(5), 274–281.
Gokhale, A. A. (1995). Collaborative learning
enhances critical thinking. Journal of Technology Education, 7(1), 22–30.
Goldfinch, J., & Raeside, R. (1990). Development
of a peer assessment technique for obtaining
individual marks on a group project. Assessment and Evaluation in Higher Education,
15(3), 210–231.
Gueldenzoph, L. E., & May, G. L. (2002). Collaborative peer evaluation: Best practices for group
member assessments. Business Communication
Quarterly, 65(1), 9–20.
Haas, A. L., Hass, R. W., & Wotruba, T. R. (1998).
The use of self-ratings and peer ratings to evaluate performances of student group members.
Journal of Marketing Education, 20(3),
200–209.
Hancock, D. R. (1995). What teachers may do to
influence student motivation: An application of
expectancy theory. The Journal of General
Education, 44(3), 171–179.
Harrell, A. M., Caldwell, C., & Doty, E. (1985).
Within-person expectancy theory predictions of
accounting students’ motivation to achieve academic success. Accounting Review, 60(4),
724–735.
Johnson, C. B., & Smith, F. I. (1997). Assessment
of a complex peer evaluation instrument for
May/June 2004
281
team learning and group processes. Accounting
Education, 2(1), 21–40.
Kramer, J. F. (1990). Perceived similarity and
accuracy of peer ratings. Journal of Educational Psychology, 82(2), 213–218.
Levi, D., & Cadiz, D. (1998). Evaluating team
work on student projects: The use of behaviorally anchored scales to evaluate student performance. ERIC Document Reproduction Service, ED 424250.
Montgomery, D. C. (1984). Design and analysis
of experiments. New York: Wiley.
Morahan-Martin, J. (1996). Should peers’ evaluations be used in class projects? Questions
regarding reliability, leniency, and acceptance.
Psychological Reports, 78(3), 1243–1250.
Murky, D., & Frizzier, K. B. (1986). A within-subjects test of expectancy theory in a public
accounting environment. Journal of Accounting
Research, 24(2), 400–404.
Sherrard, W. R., & Raafat, F. (1994). An empirical
study of peer bias in evaluations: Students rating students. Journal of Education for Business,
70(1), 43–47.
Smith, K. A. (1998). Grading cooperative projects.
New Directions for Teaching and Learning,
74(2), 59–67.
Snead, K. C., & Harrell, A. M. (1995). An appli-
cation of expectancy theory to explain a manager’s intention to use a decision support system. Decision Sciences, 25(4), 499–513.
Stahl, M. J., & Harrell, A. M. (1981). Modeling
effort decisions with behavioral decision theory: Toward an individual differences model of
expectancy theory. Organizational Behavior
and Human Performance, 27(3), 303–325.
Vroom, V. C. (1964). Work and motivation. New
York: Wiley.
Williams, D. L., Beard, J. D., & Rymer, J.
(1991). Team projects: Achieving their full
potential. Journal of Marketing Education,
13(2), 45–53.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:26 12 January 2016
APPENDIX
Instructions
As a student involved in collaborative or team-based projects, you are asked to evaluate the performance of your peers (group members). These peer evaluations may be
used in various ways, such as determining peers’ grades, improving peers’ performance and behavior, enhancing group productivity and collaboration, and reducing
conflict and uneven workload distribution.
This exercise presents 16 situations. Each situation is different with respect to how the
peer evaluation is likely to be used. We would like to know how attractive participation in such peer evaluation is to you in each given situation.
You are asked to make two decisions. You must first decide how attractive it would be
for you to participate in the peer evaluation (Decision A). Next you must decide how
much effort you would exert in completing the peer evaluation (Decision B). Use the
information provided in each situation to reach your decisions. There are no “right” or
“wrong” responses, so express your opinions freely. A sample situation is provided
below. The 16 different situations start on the next page.
EXAMPLE QUESTIONNAIRE
The likelihood that your peer evaluation feedback:
will be taken into consideration in determining
peers’ grades is …………………………...…………...……….…...…
will be provided to individual group member in
improving his or her performance and behavior is ...………..…...…
will be used to enhance productivity and collaboration
of the group in meeting its goal is ......…………...…...…...…...…...…
will be used to reduce conflict and uneven workload
distribution among group members is ..…...…...…...…...…...…...…...
HIGH (90%)
HIGH (90%)
HIGH (90%)
LOW (10%)
DECISION A: With the above outcomes and associated likelihood levels in mind,
indicate the attractiveness to you of participating in the peer evaluation.
–5
–4
–3
Very unattractive
–2
–1
0
+1
+2
+3
+4
+5
Very attractive
FURTHER INFORMATION: The peer evaluation contains several open-ended essay
questions, which will require a great deal of effort for you to complete. (As you know,
your participation in peer evaluations is voluntary. Thus you can choose to exert much
effort in the hopes of providing meaningful feedback or at the other extreme you can
do nothing.) If you exert a great deal of effort, the likelihood that the readers will find
your feedback helpful is ……………………………………………..... LOW (10%) *
DECISION B: Keeping in mind your attractiveness decision (DECISION A) and the
FURTHER INFORMATION, indicate the level of effort you would exert to participate in the peer evaluation.
0
1
Zero effort
2
3
4
5
6
7
8
9
10
Great deal of effort
* Despite your best effort to articulate your feelings, the peers may misinterpret your
feedback. Even the responses to multiple-choice questions are difficult to interpret
when the questions are designed poorly.
282
Journal of Education for Business