Manajemen | Fakultas Ekonomi Universitas Maritim Raja Ali Haji joeb.79.4.225-231
Journal of Education for Business
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Dimensions of Peer Evaluation, Overall
Satisfaction, and Overall Evaluation: An
Investigation in a Group Task Environment
Audhesh K. Paswan & Kamala Gollakota
To cite this article: Audhesh K. Paswan & Kamala Gollakota (2004) Dimensions of Peer
Evaluation, Overall Satisfaction, and Overall Evaluation: An Investigation in a Group Task
Environment, Journal of Education for Business, 79:4, 225-231, DOI: 10.3200/JOEB.79.4.225-231
To link to this article: http://dx.doi.org/10.3200/JOEB.79.4.225-231
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 68
View related articles
Citing articles: 15 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:24
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Dimensions of Peer Evaluation,
Overall Satisfaction, and
Overall Evaluation: An Investigation
in a Group Task Environment
AUDHESH K. PASWAN
University of North Texas
Denton, Texas
T
eams are used often in the workplace (Reagan & Rohrbaugh, 1990;
Saavedra & Kwun, 1993) and in educational institutions (Topping, 1998). By
working in groups, students gain additional insights, are exposed to multiple
points of view, learn from each other, and
learn important interpersonal and group
skills (Beatty, Haas, & Sciglimpaglia,
1996; Goretsky, 1985; Haas, Haas, &
Wotruba, 1998). Despite the advantages
of student teams, evaluation of individual
student contributions is a major challenge (Fiechtner & Davis, 1992). Not all
students contribute the same amount to
the task (Williams, Beard, & Rymer,
1991), and giving the same grade to all
team members makes better students feel
resentful and sends the wrong message to
slackers. However, because instructors
may not know an individual student’s
contribution, they often depend on team
members’ evaluations.
Peer evaluation has been studied in
myriad contexts: the armed forces
(Amir, Kovarsky, & Sharron, 1970;
Hollander, 1965), marketing and sales
(Mayfield, 1972; Waters & Waters,
1970), managerial performance (Kraut,
1975), and future job advancement
(Shore, Shore, & Thornton, 1992).
Despite important insights (Beatty,
Haas, & Sciglimpaglia, 1996; Ghorpade
& Lackritz, 2001; Haas, Haas, & Wotruba, 1998; Persons, 1998), little attention
KAMALA GOLLAKOTA
University of South Dakota
Vermillion, South Dakota
ABSTRACT. For this study, the
authors developed a multi-item, multidimensional scale for peer evaluation
encompassing five key dimensions:
dependability, task and maintenance
orientation, domineering behavior,
free-riding behavior, and individual
competence. They also tested the relationship between these dimensions
and two outcome variables: overall
evaluation and overall satisfaction.
Results indicate that although dependability and competence influenced
overall evaluation, task and maintenance orientation and free-riding
behavior influenced overall satisfaction. The authors also tested the scales
for stability across gender and prior
familiarity with the group member and
found them to be reasonably invariant.
has been given to the psychometric
scrutiny of the peer evaluation construct, its dimensions, antecedents, and
consequences. This is a significant limitation, given its complex and multidimensional nature and its relationship
with the efficiency and effectiveness of
a group (Duffy, Shaw, & Stark, 2000;
Hare & O’Neill, 2000; Reagan &
Rohrbaugh, 1990; Schwarzwald, Koslowsky, & Mager-Bibi, 1999).
Using the context of a graded group
project in an undergraduate business
capstone course, this study makes a
contribution by developing a peer evaluation scale and subsequently investigating its stability as well as its effect on
two key outcome variables—individual
satisfaction with the group and the overall evaluation of the group member.
Peer Evaluation
Extant literature suggests that in certain situations peers have more information and can evaluate performance more
accurately than supervisors (Barclay &
Harland, 1995; Imada, 1982). Studies in
military settings suggest that peer evaluation is a better predictor of performance than the grades received in training school or scores on psychometric
measures (Amir, Kovarsky, & Sharron,
1970; Hollander, 1954, 1957, 1965). In
industrial settings, peer evaluation was
found to predict sales (Mayfield, 1972;
Waters & Waters, 1970), managerial
performance (Kraut, 1975), and accident rates (McBain, 1970). Peer ratings
also predicted future job advancement
better than assessment center ratings
made by assessors (Shore, Shore, &
Thornton, 1992).
Despite this support for the usefulness of peer evaluation, doubts have
been raised regarding its acceptance,
bias, reliability, and even validity
(Cederblom & Lounsbury, 1980; Long,
Long, & Dobbins, 1998; MorahanMartin, 1996; Sherrard & Raafat, 1994;
Stubblebine, 2001). Haas, Haas, and
Wotruba (1998) observed an absence of
universally acceptable peer evaluation
March/April 2004
225
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
measures and proposed a multidimensional scale including such elements as
dependability, attendance, work quality,
doing a fair share of work, and getting
along, as well as overall evaluation
dimensions. Other studies have found
conscientiousness (dependability and
availability, punctuality, and participation) to be a crucial dimension of peer
evaluation (Antonioni & Park, 2001;
Barrick, Stewart, Neubert, & Mount,
1998; Haas, Haas, & Wotruba, 1998).
Early studies of groups identified
three categories of group behaviors:
task, maintenance, and individual
(Bales, 1970; Benne & Sheats, 1948).
Of these, task and maintenance behaviors relate to the effective functioning of
the group. Task behaviors include helping the group define its goals and working toward attainment of those goals
through techniques such as initiating
discussions, offering information and
opinions, elaborating from experience,
clarifying issues, summarizing ideas,
keeping the discussion on track, and
evaluating ideas presented. Maintenance behaviors include helping group
members work together by creating an
atmosphere conducive to achieving the
task, through such behaviors as encouraging reticent members to contribute,
supporting ideas, mediating group
member differences, and keeping the
group’s climate harmonious (Napier &
Greshenfeld, 1993). The third category,
individual behavior, may be dysfunctional to the group because it focuses on
meeting individual needs (Napier &
Greshenfeld, 1993). Individual behavior
may take the form of disruptive participation, domineering behavior, discounting of the importance of others’ contribution, withdrawal from the group, or
free-riding (Williams, Beard, & Rymer,
1991). Finally, we expect that group
members also will be evaluated on individual competence, such as command
over task-related knowledge and skills
(Biddle & Thomas, 1966). Thus, we formulated the following hypotheses:
H1: Peer evaluation in a group task
context is likely to be multidimensional
within five dimensions: dependability,
task orientation, maintenance orientation, domineering behavior, and freeriding behavior.
226
Journal of Education for Business
H2: Peer evaluation in a group task
context is likely to include the individual competence dimension.
Factors Influencing Peer
Evaluation
Research on distributive justice indicates that peer evaluation often forms
the basis for reward allocation using
rules based on equity (maximizing performance), equality (enhancing group
harmony), or need (seeking personal
welfare and individual development)
(Deutsch, 1975; Gomez, Shapiro, &
Kirkman, 2000; Kim, Park, & Suzuki,
1990). In an academic setting, inadequate peer contribution hurts the student’s grade, hence equity considerations may dominate reward allocations.
Consequently, dependability, taskoriented behavior, maintenance-oriented
behavior, and individual competence
factors are likely to influence positively
the grade allocated. In contrast, domineering and free-riding behavior, though
irritants that result in dissatisfaction
with the group, may not influence the
grade assigned. Because overall satisfaction cannot be tied to any particular
group member, it is unlikely to be an
important determinant.
H3: Overall evaluation (the overall
grade assigned to a peer) will
(a) be influenced positively by the dependability dimension;
(b) be influenced positively by the taskoriented dimension;
(c) be influenced positively by the
maintenance-oriented dimension;
(d not be influenced by the domineering dimension;
(e) not be influenced by the free-riding
dimension;
(f) be influenced positively by the personal competence dimension;
(g) not be influenced by the overall satisfaction with the group.
Satisfaction in a Team Context
Literature indicates that satisfaction is
influenced strongly by perceived status
consensus, progress toward group goals,
and freedom to participate (Heslin &
Dunphy, 1964; Keyton, 1991). Schoe-
necker, Martell, and Michlitsch (1997)
found domineering behavior to be associated with lower satisfaction. This negative relation is likely to become more
pronounced in academic settings where
strong norms of interaction equity exist
(Haas, Haas, & Wotruba, 1998). Thus,
we speculate that dependability and
task- and maintenance-oriented behaviors will positively influence overall satisfaction. The individual competence of
a peer, though important for success,
may not lead to overall satisfaction with
the group. In other words, equitable contribution by group members is likely to
result in satisfaction with the group, and
a bossy or a free-riding group member is
likely to lead to feelings of irritation and
dissatisfaction.
H4: Respondent’s overall satisfaction
with the group will
(a) be influenced positively by the
dependability dimension;
(b) be influenced positively by the taskoriented dimension;
(c) be influenced positively by the
maintenance-oriented dimension;
(d) be influenced negatively by the
domineering dimension;
(e) be influenced negatively by the freeriding dimension;
(f) not be influenced by the personal
competence dimension.
Peer Evaluation and
Contingent Factors
Several studies have focused on
demographic factors (Sherrard &
Raafat, 1994), diversity (Adler, 1991;
Schoenecker, Martell, & Michlitsch,
1997; Watson, Michaelsen, & Sharp,
1991), gender (Falchikov & Magin,
1997; Ghorpade & Lackritz, 2001), values (Glaman, Jones, & Rozelle, 1996;
Senger, 1971), self-esteem (Duffy,
Shaw, & Stark, 2000), and personality
(Antonioni & Park, 2001; Barrick,
Stewart, Neubert, & Mount, 1998;
Barry & Stewart, 1997; Byrne, 1971;
Miles, 1964) as contingent variables
influencing peer evaluation. In this
study, we examine two important contingent variables: prior familiarity with
a group member and gender.
We speculate that when a peer’s
poor performance negatively affects a
student’s grade, the student will disregard familiarity and focus on performance aspects. Thus, prior familiarity is
not likely to influence peer evaluation.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
H5: Peer evaluation of group members on the dimensions of (a) dependability, (b) task-oriented behavior, (c)
maintenance-oriented behavior, (d)
domineering behavior, (e) free-riding
behavior, (f) personal competence, and
(g) overall satisfaction will not be influenced by pre-existing familiarity with
the target group member.
Previous research has suggested that
peer evaluation is gender invariant
(Falchikov & Magin, 1997; Ghorpade &
Lackritz, 2001), and hence we also propose that peer evaluation will not be
influenced by the evaluator’s gender.
H6: Peer evaluation of group members on the dimensions of (a) dependability, (b) task-oriented behavior, (c)
maintenance-oriented behavior, (d)
domineering behavior, (e) free-riding
behavior, (f) personal competence, and
(g) overall satisfaction will not be influenced by the evaluator’s gender.
Method
Our study included 54 upper-division
undergraduate students in a capstone
course at an American Midwestern university. The research cohort was 58%
men and 42% women. The instructor
randomly assigned students to 12 teams
of four to six students each. Each team
worked on a semester-long comprehensive project that required extensive outof-class interaction and culminated in a
report and presentation. Each student
evaluated every other member of his or
her own team, resulting in 288 evaluations. All evaluations were completely
confidential. Students assigned every
team member a score between 0% and
100%, with higher scores indicating
greater contributions. The percentages
assigned by all peers were averaged to
obtain the overall peer evaluation percentage for a student. Without considering individual contributions, the instructor then assigned that percentage as the
final grade for the project. Therefore, a
student with an average of 100% on his
peer evaluations received a grade of
100% (or A) on the project, whereas a
lower average correspondingly lowered
the assigned project grade. The project
grade was 20% of the overall course
grade. The overall evaluation scores
ranged from 50% to 100% (mean of
97.6% and a standard deviation of 6.0%).
To eliminate possible vindictiveness,
we also dropped the lowest single peer
evaluation of each student. Thus, a student who did not receive a high score on
the project knew that more than one
individual in the group was not pleased
with his or her contributions, because
the student’s low grade could not be
blamed on any one individual. The policy also helped with the confidentiality
issue. It did not affect the averages used
in the study.
We used questions formulated from
the works of Haas, Haas, and Wotruba
(1998) and discussions with other
instructors and students to measure peer
evaluation dimensions. The final questionnaire contained 35 items for peer
evaluation and 7 items for evaluating
personal competence dimensions. We
measured the responses to peer evaluation items on a five-point Likert scale
anchored between 1 (strongly agree) and
5 (strongly disagree), and we measured
competence on a semantic differential
scale anchored between 1 (excellent)
and 5 (poor). Overall satisfaction
(judged by one item, “Overall I was satisfied working with this group”) was
measured on a five-point Likert scale
anchored between 1 (strongly agree) and
5 (strongly disagree). Overall evaluation
was operationalized by respondentassigned grades as discussed earlier.
First, we subjected the data to scale
purification processes using principal
component analysis and then we
assessed them for internal consistency
(Chronbach’s alpha) and convergent and
discriminant validity (using interitem
correlation). During this process, we
discarded 10 items because of redundancy in meaning, lack of fit with the
main factors, or imprecise wording. The
resulting factors (see Table 1) provide
support for Hypotheses 1 and 2. To test
Hypotheses 3 and 4, we used a multivariate regression analysis (see Table 2)
using composite (average) factor scores.
The first regression analysis used the
overall grade assigned as a dependent
variable, and the peer-evaluation
dimensions and overall satisfaction as
independent variables; the second
regression analysis used overall satisfaction as a dependent variable and the
peer evaluation dimensions as independent variables. Multicollinearity was
assessed through the tolerance value
and the variance inflation factor and was
found to be within acceptable levels
(Hair, Anderson, Tatham, & Black,
1998, p. 193). We tested Hypotheses 5
and 6 through ANOVA procedures (see
Tables 3 and 4).
Results
The Principal Components Analysis
(Table 1) provides support for Hypotheses 1 and 2. However, the scale items
converged onto five (instead of six) factors: behavior associated with task and
maintenance orientation, dependability,
domineering behavior, free-riding
behavior, and individual competence.
Both task and maintenance behaviors
converged onto one factor. The alpha
scores indicate high internal consistency
(all alphas were above 0.83). The
interitem correlations were higher within factor correlations than across factor
correlations, indicating acceptable levels
of convergent and discriminant validity.
Regression analysis results using overall assigned grades as the dependent variable provide partial support for Hypothesis 3 (see Table 2). Dependability and
personal competence dimensions positively influence the overall grade, which
supports Hypotheses 3a and 3f. Domineering behavior negatively influenced
the grade assigned, which supports
Hypothesis 3d, and free-riding behavior
and overall satisfaction had no influence
on the grade assigned, which supports
Hypotheses 3e and 3g. Hypotheses 3b
and 3c, associating task and maintenance
contributions with the overall grade
assigned, were not supported.
Regression results with overall satisfaction as the dependent variable provide
support only for Hypotheses 4b, 4c, 4e,
and 4f. Overall satisfaction related positively to task and maintenance behaviors,
negatively to the free-riding behavior,
and was unrelated to individual competence. However, dependability and domineering behavior did not influence
March/April 2004
227
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
TABLE 1. Peer Evaluation Dimensions: Principal Component Analysis
Rotated component matrix
F1
K6. Has conceptual knowledge of strategic management
K7. Has knowledge of strategic management tools and techniques
K1. Understands strategic management
K4. Has computer skills
K5. Has math/statistics knowledge
K2. Understands industry
K3. Has good writing skills
0.85
0.84
0.84
0.80
0.79
0.79
0.76
Q20. Was enthusiastic about the project
Q22. Encouraged others to participate
Q23. Helped resolve group conflicts
Q16. Liked the project
Q17. Thought the project was worthwhile
Q24. Helped others fit in with the group
Q21. Contributed good ideas
Q35. Provided constructive feedback
Q12. Was willing to accommodate others
Q10. Raised relevant questions
0.43
0.48
0.56
0.43
F2
F3
F4
F5
Competence
0.75
0.75
0.74
0.73
0.73
0.64
0.60
0.58
0.55
0.54
Q28. Tried to push his/her way around
Q27. Always wanted to lead, not participate
Q29. Didn’t like to be told to do things
Q33. Acted like a know-it-all
Q26. Was very critical of everyone
Q32. Was unwilling to listen to others’ input
Task and
maintenance
orientation
0.90
0.84
0.82
0.78
0.76
0.68
Q2. Came to meetings on time
Q1. Attended group meetings
Q3. Stayed for the entire meeting
Q4. Came late for meetings
Q5. Left meetings early
Domineering
behavior
0.87
0.82
0.79
–0.79
–0.72
Q9. Was present for the sake of being present
Q7. Was disruptive during the meeting
Q6. Was attentive during the meeting
Q8. Was present in body, not in mind
Dependability
–0.43
–0.76
–0.66
0.63
–0.63
% of variance explained (total = 72.15%)
Factor mean
Factor SD
Chronbach’s Alpha
19.79
1.67
0.63
0.95
Factor label
17.31
1.91
0.73
0.94
13.71
4.76
0.48
0.9
12.91
2.69
0.28
0.91
Free-riding
behavior
8.42
3.83
0.4
0.83
Notes. For peer evaluation (behavioral) dimensions, the scale used ranged from 1 (strongly agree) to 5 (strongly disagree). For individual competence, the
scale used ranged from 1 (excellent) to 5 (poor).
overall satisfaction. Thus, Hypotheses 4a
and 4d were not supported.
Results from the ANOVA analysis
(see Table 3) provide support for
Hypotheses 5a through 5g. Pre-existing
familiarity with a group member did not
influence peer evaluation dimensions
and overall satisfaction. ANOVA results
from Table 4 provide support for
Hypotheses 6b, 6c, 6d, 6e, and 6g. The
evaluator’s gender did not influence the
task and maintenance behavior, domineering behavior, free-riding behavior,
228
Journal of Education for Business
or individual competence. However,
dependability and overall satisfaction
differed across gender. The men found
their peers to be more dependable and
exhibited greater satisfaction with their
group experience than the women did;
therefore, Hypotheses 6a and 6f were
not supported.
Discussion
The findings support a multidimensional perspective of peer evaluation in
a group task environment based on elements of dependability, task and maintenance behavior, domineering behavior, free-riding behavior, and personal
competence. Contrary to our hypothesis, task and maintenance behaviors
emerged as one factor, possibly representing the effectiveness dimension. In
any case, the multidimensional nature
of peer evaluation corroborates previous research (Haas, Haas, & Wotruba,
1998; Napier & Greshenfeld, 1993). In
terms of stability, these dimensions are
TABLE 2. Peer Evaluation Dimensions, Overall Satisfaction, and Evaluation: Regression Analysis
Dependent variable
Overall satisfaction (grade assigned):
Hypotheses 3a through 3g
Beta coeff.
t value
Sig.
Independent variables
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Dependability (a)
Task and maintenance orientation (b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
R/F value/sig.
0.31
0.05
–0.14
–0.05
0.30
–0.00
0.57
5.42
0.58
–2.26
–0.83
3.58
–0.02
19.44
Overall satisfaction:
Hypotheses 4a through 4f
Beta coeff.
t statistic
Sig.
0.00
0.56
0.02
0.41
0.00
0.99
0.00
–0.09
0.25
0.09
–0.19
–0.03
N/A
0.31
–1.49
2.58
1.32
–2.74
–0.41
N/A
5.53
0.14
0.01
0.19
0.01
0.69
N/A
0.00
TABLE 3. Peer Evaluation Dimensions and Satisfaction: Invariance
Across Levels of Familiarity
(Hypotheses 5)
Dependability (a)
Task and maintenance orientation
(b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
Mean:
Familiar
Mean:
Stranger
ANOVA F
Sig.
2.63
2.70
3.21
0.07
1.76
4.77
3.89
1.63
1.38
1.93
4.76
3.81
1.68
1.42
2.58
0.09
1.95
0.27
0.22
0.11
0.76
0.16
0.60
0.64
TABLE 4. Peer Evaluation Dimensions and Satisfaction: Test for
Invariance Across Gender
(Hypotheses 6)
Dependability (a)
Task and maintenance orientation
(b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
Mean:
Male
Mean:
Female
ANOVA F
Sig.
2.64
2.74
9.27
0.00
1.89
4.74
3.81
1.67
1.35
1.94
4.77
3.86
1.66
1.57
0.26
0.29
1.14
0.04
9.64
0.61
0.59
0.29
0.85
0.00
Note. Overall satisfaction was measured on a scale ranging from 1 (very satisfied) to 5 (very dissatisfied).
relatively invariant across an evaluator’s gender and past familiarity with
group members. However, gender
influenced group members’ dependability and overall satisfaction. Possible
explanations may be that the women in
the study attached higher importance to
their academic success, were less tolerant of “goofing off,” or were more
demanding. However, more research in
this area is necessary.
Our study also suggests that some of
these factors influence the overall grade
allocated and overall satisfaction. Peerallocated grades to group members who
were dependable and competent were
higher probably because those characteristics increase a group’s accomplish-
ments. Interestingly, task and maintenance behavior, which reflects the
group dynamics, did not influence
group members’ grades. It is possible
that by the time group members evaluate each other, behaviors during the
meetings either have been forgotten or
were overlooked. What matters at that
time is whether the group fulfilled its
March/April 2004
229
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
task or not. In contrast, task and maintenance behavior significantly influence
overall satisfaction. The negative effect
of domineering behavior on the grade
allocated is not surprising. Especially in
an academic setting, where no one student has higher status, domineering
behavior is likely to be resented. Freeriding behavior was negatively related
to overall satisfaction, but not to the
overall grade. The dimensions of domineering, dependability, and personal
competence did not have a significant
influence on overall satisfaction.
Finally, a group member’s overall satisfaction with the group experience does
not influence his or her overall grade.
One possible explanation may be that,
because overall satisfaction is a reflection of what the person experiences in
the group, it might not influence the
assessment of an individual member.
Peer behavior during meetings does
influence satisfaction, but students tend
to be task- and goal-oriented when allocating grades and are not swayed by
“feel-good” emotion toward their peers.
Limitations and Future
Research Implications
Given the relatively sparse research
about peer evaluation in an academic
environment, we relied on research
from nonacademic environments. Berge
(1998) compared student teams working on class projects with workplace
teams and found similar characteristics
in successful and unsuccessful teams in
both environments. However, Berge
also observed differences between academic and workplace projects, for
instance, in terms of project duration,
formalization of leadership, and
accountability. Our study dealt with a
semester-long project with significant
grade implications for the students.
Those characteristics meet the criteria
set out by Werner and Lester (2001),
who argued that classroom teams that
work on a significant project with significant consequences would parallel
work teams. However, caution needs to
be exercised in extending the findings of
our study to projects with insignificant
grade implications.
The sampling frame is a limitation;
we relied primarily on one business cap230
Journal of Education for Business
stone course at a Midwestern business
school. Future research should investigate this phenomenon in different settings. Another limitation is the reliance
on only two outcome variables. Future
research should incorporate more outcome variables, objective measures, and
dimensions of peer evaluations.
Decision-Making Implications
A clear implication of this study is
the importance of using multi-item
scales for evaluating peers in group task
contexts in both industry and academic
settings. Such instruments are likely to
yield a more reliable and valid measure
of performance on a group assignment
than a single global measure. Global
measures are somewhat limited by their
lack of detail. Because they do not point
to specific areas of strength or weakness, they do not serve as useful development tools. A multi-item scale also
decreases errors such as the halo effect
by forcing raters to consider many
dimensions. Thus, it is likely to reduce
the bias in evaluation that might easily
occur with the use of global measures.
In an academic setting, an instructor
can consolidate the scores on various
dimensions for the class and can use the
results as pedagogical tools to illustrate
how groups, especially small ones,
function and how one can become an
effective member of the group. If an
instructor discusses in advance the
dimensions on which a student will be
evaluated, he or she is more likely to
elicit behavior conducive to effective
group performance. We can also expect
that such discussion is likely to improve
perceptions that the process of peer
evaluation is fair.
REFERENCES
Adler, N. (1991). International dimensions of
organizational behavior. Boston: PWS Kent.
Amir, Y., Kovarsky, Y., & Sharron, S. (1970). Peer
nominations as predictors of multi-stage promotions in a ramified organization. Journal of
Applied Psychology, 54, 462–469.
Antonioni, D., & Park, H. (2001, Summer). The
effects of personality similarity on peer ratings
of contextual work behaviors. Personnel Psychology, 54, 331.
Bales, R. F. (1970). Personality and interpersonal
behavior. New York: Holt, Rinehart, and
Winston.
Barclay, J. H., & Harland, L. K. (1995). Peer performance appraisals: The impact of rater com-
petence, rater location, and rating correctability
on fairness perceptions. Group and Organization Management, 20(1), 39–60.
Barrick, M. R., Stewart, G. L., Neubert, M. J., &
Mount, M. K. (1998). Relating member ability
and personality to work-team processes and
team effectiveness. Journal of Applied Psychology, 83, 377–391.
Barry, B., & Stewart, G. L. (1997). Composition,
process, and performance in self-managed
groups: The role of personality. Journal of
Applied Psychology, 82, 62–78.
Beatty, J., Haas, R., & Sciglimpaglia, D. (1996).
Using peer evaluations to assess individual performances in group class projects. Journal of
Marketing Education, 18(2), 17–28.
Benne, K. D., & Sheats, P. (1948). Functional
roles of group members. Journal of Social
Issues, 4(2), 41–49.
Berge, Z. (1998). Differences in teamwork
between postsecondary classrooms and the
workplace. Education and Training, 40(5),
194–201.
Biddle, B. J., & Thomas, E. J. (Eds.). (1966). Role
theory: Concepts and research. New York:
Wiley.
Byrne, D. (1971). The attraction paradigm. New
York: Academic Press.
Cederblom, D., & Lounsbury, J. W. (1980). An
investigation of user acceptance of peer evaluations. Personnel Psychology, 33(3), 567–579.
Deutsch, M. (1975). Equity, equality, and need:
What determines which value will be used as
the basis of distributive justice? Journal of
Social Issues, 31(3), 137–149.
Duffy, M. K., Shaw, J. D., & Stark, E. M. (2000).
Performance and satisfaction in conflicted
interdependent groups: When and how does
self-esteem make a difference? Academy of
Management Journal, 43(4), 772–783.
Falchikov, N., & Magin, D. (1997). Detecting
gender bias in peer marking of students’ group
process work. Assessment and Evaluation in
Higher Education, 22(4), 385–397.
Fiechtner, S. B., & Davis, E. A. (1992). Why some
groups fail: A survey of students’ experiences
with learning groups. In A. Goodsell, M. M.
Maher, & V. Tinto (Eds.), Collaborative learning: A sourcebook for higher education (pp.
59–74). University Park, PA: National Center
on Post-Secondary Teaching, Learning, and
Assessment.
Ghorpade, J., & Lackritz, J. R. (2001, May-June).
Peer evaluation in the classroom: A check for
sex and race/ethnicity effects. Journal of Education for Business, 76(5), 274–281.
Glaman, J. M., Jones, A. P., & Rozelle, R. (1996).
The effects of co-worker similarity on the emergence of affect in work teams. Group and Organization Management, 21(2), 192–216.
Gomez, C., Shapiro, D. L., & Kirkman, B. L.
(2000). The impact of collectivism and ingroup/out-group membership on the evaluation
generosity of team members. Academy of Management Journal, 43(6), 1097–1107.
Goretsky, M. E. (1985, Fall). Class projects as a
form of instruction. Journal of Marketing Education, 6, 33–37.
Haas, A. L, Haas, R., & Wotruba, T. R. (1998).
The use of self-ratings and peer-ratings to evaluate performances of student group members.
Journal of Marketing Education, 20(3),
200–210.
Hair, J. F., Jr., Anderson, R. E., Tatham, R. L., &
Black, W. C. (1998). Multivariate data analysis. New Jersey: Prentice Hall.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Hare, L. R., & O’Neill, K. (2000). Effectiveness
and efficiency in small academic peer groups: A
case study. Small Group Research, 31(1),
24–53.
Heslin, R., & Dunphy, D. (1964). Three dimensions of member satisfaction in small groups.
Human Relations, 17(2), 99–112.
Hollander, E. P. (1954). Peer nominations on leadership as a predictor of the pass-fail criterion in
naval air training. Journal of Applied Psychology, 38, 150–153.
Hollander, E. P. (1957). The reliability of peer
nominations under various conditions of
administration. Journal of Applied Psychology,
41, 85–90.
Hollander, E. P. (1965). Validity of peer nominations in predictive distant performance criteria.
Journal of Applied Psychology, 49, 434–438.
Imada, A. S. (1982, January). Social interaction,
observation, and stereotype as determinants of
differentiation in peer ratings. Organizational
Behavior and Human Performance, 29,
397–415.
Keyton, J. (1991). Evaluating individual group
member satisfaction as a situational variable.
Small Group Research, 22(2), 200–219.
Kim, K. I., Park, H., & Suzuki, N. (1990). Reward
allocations in the United States, Japan, and
Korea: A comparison of individualistic and collectivistic cultures. Academy of Management
Journal, 33(1), 188–198.
Kraut, A. I. (1975). Prediction of managerial success by peer and training staff ratings. Journal
of Applied Psychology, 59, 14–19.
Long, W. S., Long, E. J., & Dobbins, G. H.
(1998). Correlates of satisfaction with a peer
evaluation system: Investigation of performance levels and individual differences. Jour-
nal of Business and Psychology, 12(3),
299–312.
Mayfield, E. C. (1972). Value of peer nominations
in predicting life insurance sales performance.
Journal of Applied Psychology, 56, 319–323.
McBain, W. N. (1970). Arousal, monotony, and
accidents in line driving. Journal of Applied
Psychology, 54, 509–519.
Miles, R. E. (1964). Attitudes toward management
theory as a factor in managers’ relationships
with their supervisors. Academy of Management Journal, 7, 308–314.
Morahan-Martin, J. (1996). Should peers’ evaluations be used in class projects? Questions
regarding reliability, leniency, and acceptance.
(Special Issue). Psychological Reports, 78(3),
1243–1251.
Napier, R. W., & Greshenfeld, M. K. (1993).
Groups: Theory and experience (5th ed.).
Boston: Houghton Mifflin.
Persons, O. S. (1998). Factors influencing students’
peer evaluation in cooperative learning. Journal
of Education for Business, 73(4), 225–230.
Reagan, P., & Rohrbaugh, J. (1990). Group decision process effectiveness: A competing values
approach. Group and Organization Studies,
15(1), 20–43.
Saavedra, R., & Kwun, S. K. (1993). Peer evaluation in self-managing work groups. Journal of
Applied Psychology, 78(3), 450–462.
Schoenecker, T. S., Martell, K. D., & Michlitsch,
J. F. (1997). Diversity, performance and satisfaction in student group projects: An empirical
study. Research in Higher Education, 38(4),
479–495.
Schwarzwald, J., Koslowsky, M., & Mager-Bibi,
T. (1999). Peer ratings versus peer nominations
during training as predictors of actual perfor-
mance criteria. Journal of Applied Behavioral
Science, 35(3), 360–372.
Senger, J. (1971). Managers’ perceptions of subordinates’ competence as a function of personal
value orientations. Academy of Management
Journal, 14, 415–423.
Sherrard, W. R., & Raafat, F. (1994). An empirical
study of peer bias in evaluation: Students rating
students. Journal of Education for Business,
70(1), 43–48.
Shore, T. H., Shore, L. M., & Thornton, G. C.
(1992). Construct validity of self- and peer
evaluations of performance dimensions in an
assessment center. Journal of Applied Psychology, 77(1), 42–54.
Stubblebine, P. C. (2001). Perception and acceptance of evaluations by supervisors and peers.
Current Psychology, 20(1), 85–94.
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of
Educational Research, 68(3), 249–276.
Watson, W. E., Michaelsen, L. K., & Sharp, W.
(1991). Member competence, group interaction, and group decision-making: A longitudinal study. Journal of Applied Psychology, 76(6),
803–809.
Waters, L. K., & Waters, C. S. (1970). Peer nominations as predictors of short-term sales performance. Journal of Applied Psychology, 54,
42–44.
Werner, J. M., and Lester, S. (2001). Applying a
team effectiveness framework to the performance of student case teams. Human Resource
Development Quarterly, 12(4), 385–402.
Williams, D. L., Beard, J. D., & Rymer, J. (1991,
Summer). Team projects: Achieving their full
potential. Journal of Marketing Education, 13,
45–53.
March/April 2004
231
ISSN: 0883-2323 (Print) 1940-3356 (Online) Journal homepage: http://www.tandfonline.com/loi/vjeb20
Dimensions of Peer Evaluation, Overall
Satisfaction, and Overall Evaluation: An
Investigation in a Group Task Environment
Audhesh K. Paswan & Kamala Gollakota
To cite this article: Audhesh K. Paswan & Kamala Gollakota (2004) Dimensions of Peer
Evaluation, Overall Satisfaction, and Overall Evaluation: An Investigation in a Group Task
Environment, Journal of Education for Business, 79:4, 225-231, DOI: 10.3200/JOEB.79.4.225-231
To link to this article: http://dx.doi.org/10.3200/JOEB.79.4.225-231
Published online: 07 Aug 2010.
Submit your article to this journal
Article views: 68
View related articles
Citing articles: 15 View citing articles
Full Terms & Conditions of access and use can be found at
http://www.tandfonline.com/action/journalInformation?journalCode=vjeb20
Download by: [Universitas Maritim Raja Ali Haji]
Date: 12 January 2016, At: 23:24
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Dimensions of Peer Evaluation,
Overall Satisfaction, and
Overall Evaluation: An Investigation
in a Group Task Environment
AUDHESH K. PASWAN
University of North Texas
Denton, Texas
T
eams are used often in the workplace (Reagan & Rohrbaugh, 1990;
Saavedra & Kwun, 1993) and in educational institutions (Topping, 1998). By
working in groups, students gain additional insights, are exposed to multiple
points of view, learn from each other, and
learn important interpersonal and group
skills (Beatty, Haas, & Sciglimpaglia,
1996; Goretsky, 1985; Haas, Haas, &
Wotruba, 1998). Despite the advantages
of student teams, evaluation of individual
student contributions is a major challenge (Fiechtner & Davis, 1992). Not all
students contribute the same amount to
the task (Williams, Beard, & Rymer,
1991), and giving the same grade to all
team members makes better students feel
resentful and sends the wrong message to
slackers. However, because instructors
may not know an individual student’s
contribution, they often depend on team
members’ evaluations.
Peer evaluation has been studied in
myriad contexts: the armed forces
(Amir, Kovarsky, & Sharron, 1970;
Hollander, 1965), marketing and sales
(Mayfield, 1972; Waters & Waters,
1970), managerial performance (Kraut,
1975), and future job advancement
(Shore, Shore, & Thornton, 1992).
Despite important insights (Beatty,
Haas, & Sciglimpaglia, 1996; Ghorpade
& Lackritz, 2001; Haas, Haas, & Wotruba, 1998; Persons, 1998), little attention
KAMALA GOLLAKOTA
University of South Dakota
Vermillion, South Dakota
ABSTRACT. For this study, the
authors developed a multi-item, multidimensional scale for peer evaluation
encompassing five key dimensions:
dependability, task and maintenance
orientation, domineering behavior,
free-riding behavior, and individual
competence. They also tested the relationship between these dimensions
and two outcome variables: overall
evaluation and overall satisfaction.
Results indicate that although dependability and competence influenced
overall evaluation, task and maintenance orientation and free-riding
behavior influenced overall satisfaction. The authors also tested the scales
for stability across gender and prior
familiarity with the group member and
found them to be reasonably invariant.
has been given to the psychometric
scrutiny of the peer evaluation construct, its dimensions, antecedents, and
consequences. This is a significant limitation, given its complex and multidimensional nature and its relationship
with the efficiency and effectiveness of
a group (Duffy, Shaw, & Stark, 2000;
Hare & O’Neill, 2000; Reagan &
Rohrbaugh, 1990; Schwarzwald, Koslowsky, & Mager-Bibi, 1999).
Using the context of a graded group
project in an undergraduate business
capstone course, this study makes a
contribution by developing a peer evaluation scale and subsequently investigating its stability as well as its effect on
two key outcome variables—individual
satisfaction with the group and the overall evaluation of the group member.
Peer Evaluation
Extant literature suggests that in certain situations peers have more information and can evaluate performance more
accurately than supervisors (Barclay &
Harland, 1995; Imada, 1982). Studies in
military settings suggest that peer evaluation is a better predictor of performance than the grades received in training school or scores on psychometric
measures (Amir, Kovarsky, & Sharron,
1970; Hollander, 1954, 1957, 1965). In
industrial settings, peer evaluation was
found to predict sales (Mayfield, 1972;
Waters & Waters, 1970), managerial
performance (Kraut, 1975), and accident rates (McBain, 1970). Peer ratings
also predicted future job advancement
better than assessment center ratings
made by assessors (Shore, Shore, &
Thornton, 1992).
Despite this support for the usefulness of peer evaluation, doubts have
been raised regarding its acceptance,
bias, reliability, and even validity
(Cederblom & Lounsbury, 1980; Long,
Long, & Dobbins, 1998; MorahanMartin, 1996; Sherrard & Raafat, 1994;
Stubblebine, 2001). Haas, Haas, and
Wotruba (1998) observed an absence of
universally acceptable peer evaluation
March/April 2004
225
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
measures and proposed a multidimensional scale including such elements as
dependability, attendance, work quality,
doing a fair share of work, and getting
along, as well as overall evaluation
dimensions. Other studies have found
conscientiousness (dependability and
availability, punctuality, and participation) to be a crucial dimension of peer
evaluation (Antonioni & Park, 2001;
Barrick, Stewart, Neubert, & Mount,
1998; Haas, Haas, & Wotruba, 1998).
Early studies of groups identified
three categories of group behaviors:
task, maintenance, and individual
(Bales, 1970; Benne & Sheats, 1948).
Of these, task and maintenance behaviors relate to the effective functioning of
the group. Task behaviors include helping the group define its goals and working toward attainment of those goals
through techniques such as initiating
discussions, offering information and
opinions, elaborating from experience,
clarifying issues, summarizing ideas,
keeping the discussion on track, and
evaluating ideas presented. Maintenance behaviors include helping group
members work together by creating an
atmosphere conducive to achieving the
task, through such behaviors as encouraging reticent members to contribute,
supporting ideas, mediating group
member differences, and keeping the
group’s climate harmonious (Napier &
Greshenfeld, 1993). The third category,
individual behavior, may be dysfunctional to the group because it focuses on
meeting individual needs (Napier &
Greshenfeld, 1993). Individual behavior
may take the form of disruptive participation, domineering behavior, discounting of the importance of others’ contribution, withdrawal from the group, or
free-riding (Williams, Beard, & Rymer,
1991). Finally, we expect that group
members also will be evaluated on individual competence, such as command
over task-related knowledge and skills
(Biddle & Thomas, 1966). Thus, we formulated the following hypotheses:
H1: Peer evaluation in a group task
context is likely to be multidimensional
within five dimensions: dependability,
task orientation, maintenance orientation, domineering behavior, and freeriding behavior.
226
Journal of Education for Business
H2: Peer evaluation in a group task
context is likely to include the individual competence dimension.
Factors Influencing Peer
Evaluation
Research on distributive justice indicates that peer evaluation often forms
the basis for reward allocation using
rules based on equity (maximizing performance), equality (enhancing group
harmony), or need (seeking personal
welfare and individual development)
(Deutsch, 1975; Gomez, Shapiro, &
Kirkman, 2000; Kim, Park, & Suzuki,
1990). In an academic setting, inadequate peer contribution hurts the student’s grade, hence equity considerations may dominate reward allocations.
Consequently, dependability, taskoriented behavior, maintenance-oriented
behavior, and individual competence
factors are likely to influence positively
the grade allocated. In contrast, domineering and free-riding behavior, though
irritants that result in dissatisfaction
with the group, may not influence the
grade assigned. Because overall satisfaction cannot be tied to any particular
group member, it is unlikely to be an
important determinant.
H3: Overall evaluation (the overall
grade assigned to a peer) will
(a) be influenced positively by the dependability dimension;
(b) be influenced positively by the taskoriented dimension;
(c) be influenced positively by the
maintenance-oriented dimension;
(d not be influenced by the domineering dimension;
(e) not be influenced by the free-riding
dimension;
(f) be influenced positively by the personal competence dimension;
(g) not be influenced by the overall satisfaction with the group.
Satisfaction in a Team Context
Literature indicates that satisfaction is
influenced strongly by perceived status
consensus, progress toward group goals,
and freedom to participate (Heslin &
Dunphy, 1964; Keyton, 1991). Schoe-
necker, Martell, and Michlitsch (1997)
found domineering behavior to be associated with lower satisfaction. This negative relation is likely to become more
pronounced in academic settings where
strong norms of interaction equity exist
(Haas, Haas, & Wotruba, 1998). Thus,
we speculate that dependability and
task- and maintenance-oriented behaviors will positively influence overall satisfaction. The individual competence of
a peer, though important for success,
may not lead to overall satisfaction with
the group. In other words, equitable contribution by group members is likely to
result in satisfaction with the group, and
a bossy or a free-riding group member is
likely to lead to feelings of irritation and
dissatisfaction.
H4: Respondent’s overall satisfaction
with the group will
(a) be influenced positively by the
dependability dimension;
(b) be influenced positively by the taskoriented dimension;
(c) be influenced positively by the
maintenance-oriented dimension;
(d) be influenced negatively by the
domineering dimension;
(e) be influenced negatively by the freeriding dimension;
(f) not be influenced by the personal
competence dimension.
Peer Evaluation and
Contingent Factors
Several studies have focused on
demographic factors (Sherrard &
Raafat, 1994), diversity (Adler, 1991;
Schoenecker, Martell, & Michlitsch,
1997; Watson, Michaelsen, & Sharp,
1991), gender (Falchikov & Magin,
1997; Ghorpade & Lackritz, 2001), values (Glaman, Jones, & Rozelle, 1996;
Senger, 1971), self-esteem (Duffy,
Shaw, & Stark, 2000), and personality
(Antonioni & Park, 2001; Barrick,
Stewart, Neubert, & Mount, 1998;
Barry & Stewart, 1997; Byrne, 1971;
Miles, 1964) as contingent variables
influencing peer evaluation. In this
study, we examine two important contingent variables: prior familiarity with
a group member and gender.
We speculate that when a peer’s
poor performance negatively affects a
student’s grade, the student will disregard familiarity and focus on performance aspects. Thus, prior familiarity is
not likely to influence peer evaluation.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
H5: Peer evaluation of group members on the dimensions of (a) dependability, (b) task-oriented behavior, (c)
maintenance-oriented behavior, (d)
domineering behavior, (e) free-riding
behavior, (f) personal competence, and
(g) overall satisfaction will not be influenced by pre-existing familiarity with
the target group member.
Previous research has suggested that
peer evaluation is gender invariant
(Falchikov & Magin, 1997; Ghorpade &
Lackritz, 2001), and hence we also propose that peer evaluation will not be
influenced by the evaluator’s gender.
H6: Peer evaluation of group members on the dimensions of (a) dependability, (b) task-oriented behavior, (c)
maintenance-oriented behavior, (d)
domineering behavior, (e) free-riding
behavior, (f) personal competence, and
(g) overall satisfaction will not be influenced by the evaluator’s gender.
Method
Our study included 54 upper-division
undergraduate students in a capstone
course at an American Midwestern university. The research cohort was 58%
men and 42% women. The instructor
randomly assigned students to 12 teams
of four to six students each. Each team
worked on a semester-long comprehensive project that required extensive outof-class interaction and culminated in a
report and presentation. Each student
evaluated every other member of his or
her own team, resulting in 288 evaluations. All evaluations were completely
confidential. Students assigned every
team member a score between 0% and
100%, with higher scores indicating
greater contributions. The percentages
assigned by all peers were averaged to
obtain the overall peer evaluation percentage for a student. Without considering individual contributions, the instructor then assigned that percentage as the
final grade for the project. Therefore, a
student with an average of 100% on his
peer evaluations received a grade of
100% (or A) on the project, whereas a
lower average correspondingly lowered
the assigned project grade. The project
grade was 20% of the overall course
grade. The overall evaluation scores
ranged from 50% to 100% (mean of
97.6% and a standard deviation of 6.0%).
To eliminate possible vindictiveness,
we also dropped the lowest single peer
evaluation of each student. Thus, a student who did not receive a high score on
the project knew that more than one
individual in the group was not pleased
with his or her contributions, because
the student’s low grade could not be
blamed on any one individual. The policy also helped with the confidentiality
issue. It did not affect the averages used
in the study.
We used questions formulated from
the works of Haas, Haas, and Wotruba
(1998) and discussions with other
instructors and students to measure peer
evaluation dimensions. The final questionnaire contained 35 items for peer
evaluation and 7 items for evaluating
personal competence dimensions. We
measured the responses to peer evaluation items on a five-point Likert scale
anchored between 1 (strongly agree) and
5 (strongly disagree), and we measured
competence on a semantic differential
scale anchored between 1 (excellent)
and 5 (poor). Overall satisfaction
(judged by one item, “Overall I was satisfied working with this group”) was
measured on a five-point Likert scale
anchored between 1 (strongly agree) and
5 (strongly disagree). Overall evaluation
was operationalized by respondentassigned grades as discussed earlier.
First, we subjected the data to scale
purification processes using principal
component analysis and then we
assessed them for internal consistency
(Chronbach’s alpha) and convergent and
discriminant validity (using interitem
correlation). During this process, we
discarded 10 items because of redundancy in meaning, lack of fit with the
main factors, or imprecise wording. The
resulting factors (see Table 1) provide
support for Hypotheses 1 and 2. To test
Hypotheses 3 and 4, we used a multivariate regression analysis (see Table 2)
using composite (average) factor scores.
The first regression analysis used the
overall grade assigned as a dependent
variable, and the peer-evaluation
dimensions and overall satisfaction as
independent variables; the second
regression analysis used overall satisfaction as a dependent variable and the
peer evaluation dimensions as independent variables. Multicollinearity was
assessed through the tolerance value
and the variance inflation factor and was
found to be within acceptable levels
(Hair, Anderson, Tatham, & Black,
1998, p. 193). We tested Hypotheses 5
and 6 through ANOVA procedures (see
Tables 3 and 4).
Results
The Principal Components Analysis
(Table 1) provides support for Hypotheses 1 and 2. However, the scale items
converged onto five (instead of six) factors: behavior associated with task and
maintenance orientation, dependability,
domineering behavior, free-riding
behavior, and individual competence.
Both task and maintenance behaviors
converged onto one factor. The alpha
scores indicate high internal consistency
(all alphas were above 0.83). The
interitem correlations were higher within factor correlations than across factor
correlations, indicating acceptable levels
of convergent and discriminant validity.
Regression analysis results using overall assigned grades as the dependent variable provide partial support for Hypothesis 3 (see Table 2). Dependability and
personal competence dimensions positively influence the overall grade, which
supports Hypotheses 3a and 3f. Domineering behavior negatively influenced
the grade assigned, which supports
Hypothesis 3d, and free-riding behavior
and overall satisfaction had no influence
on the grade assigned, which supports
Hypotheses 3e and 3g. Hypotheses 3b
and 3c, associating task and maintenance
contributions with the overall grade
assigned, were not supported.
Regression results with overall satisfaction as the dependent variable provide
support only for Hypotheses 4b, 4c, 4e,
and 4f. Overall satisfaction related positively to task and maintenance behaviors,
negatively to the free-riding behavior,
and was unrelated to individual competence. However, dependability and domineering behavior did not influence
March/April 2004
227
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
TABLE 1. Peer Evaluation Dimensions: Principal Component Analysis
Rotated component matrix
F1
K6. Has conceptual knowledge of strategic management
K7. Has knowledge of strategic management tools and techniques
K1. Understands strategic management
K4. Has computer skills
K5. Has math/statistics knowledge
K2. Understands industry
K3. Has good writing skills
0.85
0.84
0.84
0.80
0.79
0.79
0.76
Q20. Was enthusiastic about the project
Q22. Encouraged others to participate
Q23. Helped resolve group conflicts
Q16. Liked the project
Q17. Thought the project was worthwhile
Q24. Helped others fit in with the group
Q21. Contributed good ideas
Q35. Provided constructive feedback
Q12. Was willing to accommodate others
Q10. Raised relevant questions
0.43
0.48
0.56
0.43
F2
F3
F4
F5
Competence
0.75
0.75
0.74
0.73
0.73
0.64
0.60
0.58
0.55
0.54
Q28. Tried to push his/her way around
Q27. Always wanted to lead, not participate
Q29. Didn’t like to be told to do things
Q33. Acted like a know-it-all
Q26. Was very critical of everyone
Q32. Was unwilling to listen to others’ input
Task and
maintenance
orientation
0.90
0.84
0.82
0.78
0.76
0.68
Q2. Came to meetings on time
Q1. Attended group meetings
Q3. Stayed for the entire meeting
Q4. Came late for meetings
Q5. Left meetings early
Domineering
behavior
0.87
0.82
0.79
–0.79
–0.72
Q9. Was present for the sake of being present
Q7. Was disruptive during the meeting
Q6. Was attentive during the meeting
Q8. Was present in body, not in mind
Dependability
–0.43
–0.76
–0.66
0.63
–0.63
% of variance explained (total = 72.15%)
Factor mean
Factor SD
Chronbach’s Alpha
19.79
1.67
0.63
0.95
Factor label
17.31
1.91
0.73
0.94
13.71
4.76
0.48
0.9
12.91
2.69
0.28
0.91
Free-riding
behavior
8.42
3.83
0.4
0.83
Notes. For peer evaluation (behavioral) dimensions, the scale used ranged from 1 (strongly agree) to 5 (strongly disagree). For individual competence, the
scale used ranged from 1 (excellent) to 5 (poor).
overall satisfaction. Thus, Hypotheses 4a
and 4d were not supported.
Results from the ANOVA analysis
(see Table 3) provide support for
Hypotheses 5a through 5g. Pre-existing
familiarity with a group member did not
influence peer evaluation dimensions
and overall satisfaction. ANOVA results
from Table 4 provide support for
Hypotheses 6b, 6c, 6d, 6e, and 6g. The
evaluator’s gender did not influence the
task and maintenance behavior, domineering behavior, free-riding behavior,
228
Journal of Education for Business
or individual competence. However,
dependability and overall satisfaction
differed across gender. The men found
their peers to be more dependable and
exhibited greater satisfaction with their
group experience than the women did;
therefore, Hypotheses 6a and 6f were
not supported.
Discussion
The findings support a multidimensional perspective of peer evaluation in
a group task environment based on elements of dependability, task and maintenance behavior, domineering behavior, free-riding behavior, and personal
competence. Contrary to our hypothesis, task and maintenance behaviors
emerged as one factor, possibly representing the effectiveness dimension. In
any case, the multidimensional nature
of peer evaluation corroborates previous research (Haas, Haas, & Wotruba,
1998; Napier & Greshenfeld, 1993). In
terms of stability, these dimensions are
TABLE 2. Peer Evaluation Dimensions, Overall Satisfaction, and Evaluation: Regression Analysis
Dependent variable
Overall satisfaction (grade assigned):
Hypotheses 3a through 3g
Beta coeff.
t value
Sig.
Independent variables
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Dependability (a)
Task and maintenance orientation (b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
R/F value/sig.
0.31
0.05
–0.14
–0.05
0.30
–0.00
0.57
5.42
0.58
–2.26
–0.83
3.58
–0.02
19.44
Overall satisfaction:
Hypotheses 4a through 4f
Beta coeff.
t statistic
Sig.
0.00
0.56
0.02
0.41
0.00
0.99
0.00
–0.09
0.25
0.09
–0.19
–0.03
N/A
0.31
–1.49
2.58
1.32
–2.74
–0.41
N/A
5.53
0.14
0.01
0.19
0.01
0.69
N/A
0.00
TABLE 3. Peer Evaluation Dimensions and Satisfaction: Invariance
Across Levels of Familiarity
(Hypotheses 5)
Dependability (a)
Task and maintenance orientation
(b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
Mean:
Familiar
Mean:
Stranger
ANOVA F
Sig.
2.63
2.70
3.21
0.07
1.76
4.77
3.89
1.63
1.38
1.93
4.76
3.81
1.68
1.42
2.58
0.09
1.95
0.27
0.22
0.11
0.76
0.16
0.60
0.64
TABLE 4. Peer Evaluation Dimensions and Satisfaction: Test for
Invariance Across Gender
(Hypotheses 6)
Dependability (a)
Task and maintenance orientation
(b/c)
Domineering behavior (d)
Free-riding behavior (e)
Competence (f)
Overall satisfaction (g)
Mean:
Male
Mean:
Female
ANOVA F
Sig.
2.64
2.74
9.27
0.00
1.89
4.74
3.81
1.67
1.35
1.94
4.77
3.86
1.66
1.57
0.26
0.29
1.14
0.04
9.64
0.61
0.59
0.29
0.85
0.00
Note. Overall satisfaction was measured on a scale ranging from 1 (very satisfied) to 5 (very dissatisfied).
relatively invariant across an evaluator’s gender and past familiarity with
group members. However, gender
influenced group members’ dependability and overall satisfaction. Possible
explanations may be that the women in
the study attached higher importance to
their academic success, were less tolerant of “goofing off,” or were more
demanding. However, more research in
this area is necessary.
Our study also suggests that some of
these factors influence the overall grade
allocated and overall satisfaction. Peerallocated grades to group members who
were dependable and competent were
higher probably because those characteristics increase a group’s accomplish-
ments. Interestingly, task and maintenance behavior, which reflects the
group dynamics, did not influence
group members’ grades. It is possible
that by the time group members evaluate each other, behaviors during the
meetings either have been forgotten or
were overlooked. What matters at that
time is whether the group fulfilled its
March/April 2004
229
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
task or not. In contrast, task and maintenance behavior significantly influence
overall satisfaction. The negative effect
of domineering behavior on the grade
allocated is not surprising. Especially in
an academic setting, where no one student has higher status, domineering
behavior is likely to be resented. Freeriding behavior was negatively related
to overall satisfaction, but not to the
overall grade. The dimensions of domineering, dependability, and personal
competence did not have a significant
influence on overall satisfaction.
Finally, a group member’s overall satisfaction with the group experience does
not influence his or her overall grade.
One possible explanation may be that,
because overall satisfaction is a reflection of what the person experiences in
the group, it might not influence the
assessment of an individual member.
Peer behavior during meetings does
influence satisfaction, but students tend
to be task- and goal-oriented when allocating grades and are not swayed by
“feel-good” emotion toward their peers.
Limitations and Future
Research Implications
Given the relatively sparse research
about peer evaluation in an academic
environment, we relied on research
from nonacademic environments. Berge
(1998) compared student teams working on class projects with workplace
teams and found similar characteristics
in successful and unsuccessful teams in
both environments. However, Berge
also observed differences between academic and workplace projects, for
instance, in terms of project duration,
formalization of leadership, and
accountability. Our study dealt with a
semester-long project with significant
grade implications for the students.
Those characteristics meet the criteria
set out by Werner and Lester (2001),
who argued that classroom teams that
work on a significant project with significant consequences would parallel
work teams. However, caution needs to
be exercised in extending the findings of
our study to projects with insignificant
grade implications.
The sampling frame is a limitation;
we relied primarily on one business cap230
Journal of Education for Business
stone course at a Midwestern business
school. Future research should investigate this phenomenon in different settings. Another limitation is the reliance
on only two outcome variables. Future
research should incorporate more outcome variables, objective measures, and
dimensions of peer evaluations.
Decision-Making Implications
A clear implication of this study is
the importance of using multi-item
scales for evaluating peers in group task
contexts in both industry and academic
settings. Such instruments are likely to
yield a more reliable and valid measure
of performance on a group assignment
than a single global measure. Global
measures are somewhat limited by their
lack of detail. Because they do not point
to specific areas of strength or weakness, they do not serve as useful development tools. A multi-item scale also
decreases errors such as the halo effect
by forcing raters to consider many
dimensions. Thus, it is likely to reduce
the bias in evaluation that might easily
occur with the use of global measures.
In an academic setting, an instructor
can consolidate the scores on various
dimensions for the class and can use the
results as pedagogical tools to illustrate
how groups, especially small ones,
function and how one can become an
effective member of the group. If an
instructor discusses in advance the
dimensions on which a student will be
evaluated, he or she is more likely to
elicit behavior conducive to effective
group performance. We can also expect
that such discussion is likely to improve
perceptions that the process of peer
evaluation is fair.
REFERENCES
Adler, N. (1991). International dimensions of
organizational behavior. Boston: PWS Kent.
Amir, Y., Kovarsky, Y., & Sharron, S. (1970). Peer
nominations as predictors of multi-stage promotions in a ramified organization. Journal of
Applied Psychology, 54, 462–469.
Antonioni, D., & Park, H. (2001, Summer). The
effects of personality similarity on peer ratings
of contextual work behaviors. Personnel Psychology, 54, 331.
Bales, R. F. (1970). Personality and interpersonal
behavior. New York: Holt, Rinehart, and
Winston.
Barclay, J. H., & Harland, L. K. (1995). Peer performance appraisals: The impact of rater com-
petence, rater location, and rating correctability
on fairness perceptions. Group and Organization Management, 20(1), 39–60.
Barrick, M. R., Stewart, G. L., Neubert, M. J., &
Mount, M. K. (1998). Relating member ability
and personality to work-team processes and
team effectiveness. Journal of Applied Psychology, 83, 377–391.
Barry, B., & Stewart, G. L. (1997). Composition,
process, and performance in self-managed
groups: The role of personality. Journal of
Applied Psychology, 82, 62–78.
Beatty, J., Haas, R., & Sciglimpaglia, D. (1996).
Using peer evaluations to assess individual performances in group class projects. Journal of
Marketing Education, 18(2), 17–28.
Benne, K. D., & Sheats, P. (1948). Functional
roles of group members. Journal of Social
Issues, 4(2), 41–49.
Berge, Z. (1998). Differences in teamwork
between postsecondary classrooms and the
workplace. Education and Training, 40(5),
194–201.
Biddle, B. J., & Thomas, E. J. (Eds.). (1966). Role
theory: Concepts and research. New York:
Wiley.
Byrne, D. (1971). The attraction paradigm. New
York: Academic Press.
Cederblom, D., & Lounsbury, J. W. (1980). An
investigation of user acceptance of peer evaluations. Personnel Psychology, 33(3), 567–579.
Deutsch, M. (1975). Equity, equality, and need:
What determines which value will be used as
the basis of distributive justice? Journal of
Social Issues, 31(3), 137–149.
Duffy, M. K., Shaw, J. D., & Stark, E. M. (2000).
Performance and satisfaction in conflicted
interdependent groups: When and how does
self-esteem make a difference? Academy of
Management Journal, 43(4), 772–783.
Falchikov, N., & Magin, D. (1997). Detecting
gender bias in peer marking of students’ group
process work. Assessment and Evaluation in
Higher Education, 22(4), 385–397.
Fiechtner, S. B., & Davis, E. A. (1992). Why some
groups fail: A survey of students’ experiences
with learning groups. In A. Goodsell, M. M.
Maher, & V. Tinto (Eds.), Collaborative learning: A sourcebook for higher education (pp.
59–74). University Park, PA: National Center
on Post-Secondary Teaching, Learning, and
Assessment.
Ghorpade, J., & Lackritz, J. R. (2001, May-June).
Peer evaluation in the classroom: A check for
sex and race/ethnicity effects. Journal of Education for Business, 76(5), 274–281.
Glaman, J. M., Jones, A. P., & Rozelle, R. (1996).
The effects of co-worker similarity on the emergence of affect in work teams. Group and Organization Management, 21(2), 192–216.
Gomez, C., Shapiro, D. L., & Kirkman, B. L.
(2000). The impact of collectivism and ingroup/out-group membership on the evaluation
generosity of team members. Academy of Management Journal, 43(6), 1097–1107.
Goretsky, M. E. (1985, Fall). Class projects as a
form of instruction. Journal of Marketing Education, 6, 33–37.
Haas, A. L, Haas, R., & Wotruba, T. R. (1998).
The use of self-ratings and peer-ratings to evaluate performances of student group members.
Journal of Marketing Education, 20(3),
200–210.
Hair, J. F., Jr., Anderson, R. E., Tatham, R. L., &
Black, W. C. (1998). Multivariate data analysis. New Jersey: Prentice Hall.
Downloaded by [Universitas Maritim Raja Ali Haji] at 23:24 12 January 2016
Hare, L. R., & O’Neill, K. (2000). Effectiveness
and efficiency in small academic peer groups: A
case study. Small Group Research, 31(1),
24–53.
Heslin, R., & Dunphy, D. (1964). Three dimensions of member satisfaction in small groups.
Human Relations, 17(2), 99–112.
Hollander, E. P. (1954). Peer nominations on leadership as a predictor of the pass-fail criterion in
naval air training. Journal of Applied Psychology, 38, 150–153.
Hollander, E. P. (1957). The reliability of peer
nominations under various conditions of
administration. Journal of Applied Psychology,
41, 85–90.
Hollander, E. P. (1965). Validity of peer nominations in predictive distant performance criteria.
Journal of Applied Psychology, 49, 434–438.
Imada, A. S. (1982, January). Social interaction,
observation, and stereotype as determinants of
differentiation in peer ratings. Organizational
Behavior and Human Performance, 29,
397–415.
Keyton, J. (1991). Evaluating individual group
member satisfaction as a situational variable.
Small Group Research, 22(2), 200–219.
Kim, K. I., Park, H., & Suzuki, N. (1990). Reward
allocations in the United States, Japan, and
Korea: A comparison of individualistic and collectivistic cultures. Academy of Management
Journal, 33(1), 188–198.
Kraut, A. I. (1975). Prediction of managerial success by peer and training staff ratings. Journal
of Applied Psychology, 59, 14–19.
Long, W. S., Long, E. J., & Dobbins, G. H.
(1998). Correlates of satisfaction with a peer
evaluation system: Investigation of performance levels and individual differences. Jour-
nal of Business and Psychology, 12(3),
299–312.
Mayfield, E. C. (1972). Value of peer nominations
in predicting life insurance sales performance.
Journal of Applied Psychology, 56, 319–323.
McBain, W. N. (1970). Arousal, monotony, and
accidents in line driving. Journal of Applied
Psychology, 54, 509–519.
Miles, R. E. (1964). Attitudes toward management
theory as a factor in managers’ relationships
with their supervisors. Academy of Management Journal, 7, 308–314.
Morahan-Martin, J. (1996). Should peers’ evaluations be used in class projects? Questions
regarding reliability, leniency, and acceptance.
(Special Issue). Psychological Reports, 78(3),
1243–1251.
Napier, R. W., & Greshenfeld, M. K. (1993).
Groups: Theory and experience (5th ed.).
Boston: Houghton Mifflin.
Persons, O. S. (1998). Factors influencing students’
peer evaluation in cooperative learning. Journal
of Education for Business, 73(4), 225–230.
Reagan, P., & Rohrbaugh, J. (1990). Group decision process effectiveness: A competing values
approach. Group and Organization Studies,
15(1), 20–43.
Saavedra, R., & Kwun, S. K. (1993). Peer evaluation in self-managing work groups. Journal of
Applied Psychology, 78(3), 450–462.
Schoenecker, T. S., Martell, K. D., & Michlitsch,
J. F. (1997). Diversity, performance and satisfaction in student group projects: An empirical
study. Research in Higher Education, 38(4),
479–495.
Schwarzwald, J., Koslowsky, M., & Mager-Bibi,
T. (1999). Peer ratings versus peer nominations
during training as predictors of actual perfor-
mance criteria. Journal of Applied Behavioral
Science, 35(3), 360–372.
Senger, J. (1971). Managers’ perceptions of subordinates’ competence as a function of personal
value orientations. Academy of Management
Journal, 14, 415–423.
Sherrard, W. R., & Raafat, F. (1994). An empirical
study of peer bias in evaluation: Students rating
students. Journal of Education for Business,
70(1), 43–48.
Shore, T. H., Shore, L. M., & Thornton, G. C.
(1992). Construct validity of self- and peer
evaluations of performance dimensions in an
assessment center. Journal of Applied Psychology, 77(1), 42–54.
Stubblebine, P. C. (2001). Perception and acceptance of evaluations by supervisors and peers.
Current Psychology, 20(1), 85–94.
Topping, K. (1998). Peer assessment between students in colleges and universities. Review of
Educational Research, 68(3), 249–276.
Watson, W. E., Michaelsen, L. K., & Sharp, W.
(1991). Member competence, group interaction, and group decision-making: A longitudinal study. Journal of Applied Psychology, 76(6),
803–809.
Waters, L. K., & Waters, C. S. (1970). Peer nominations as predictors of short-term sales performance. Journal of Applied Psychology, 54,
42–44.
Werner, J. M., and Lester, S. (2001). Applying a
team effectiveness framework to the performance of student case teams. Human Resource
Development Quarterly, 12(4), 385–402.
Williams, D. L., Beard, J. D., & Rymer, J. (1991,
Summer). Team projects: Achieving their full
potential. Journal of Marketing Education, 13,
45–53.
March/April 2004
231