presentation_3036_27527.ppt 1143KB Jun 23 2011 12:33:10 PM
Department of Technology Education and Digital Systems
University of Piraeus
Authentic Assessment in Computer
Supported Collaborative Learning
Environments in Higher Education
Konstantinos Mathiopoulos
Fotini Paraskeva
University of Piraeus, Department of Technology Education and
Digital Systems
Konstantinos Mathiopoulos
Fotini Paraskeva
1
Department of Technology Education and Digital Systems
University of Piraeus
Contents at a glance
•
•
•
•
•
•
•
•
Theoretical background of Authentic Assessment
Definitions of Authentic Assessment
Traditional Assessment VS Authentic Assessment
Advantages and disadvantages of Authentic Assessment
Authentic Assessment’s expectations
Research gap
Goal of the paper
Study on Authentic Assessment in Computer Supported
Collaborative Learning Environments in Higher Education
• Conclusions
• Future research
Konstantinos Mathiopoulos
Fotini Paraskeva
2
Department of Technology Education and Digital Systems
University of Piraeus
Konstantinos Mathiopoulos
Fotini Paraskeva
3
Department of Technology Education and Digital Systems
University of Piraeus
Definitions of Authentic Assessment:
"A form of assessment in which students are asked to perform real-world
tasks that demonstrate meaningful application of essential knowledge and
skills "-- Jon Mueller 2008
"...Engaging and worthy problems or questions of importance, in which
students must use knowledge to fashion performances effectively and
creatively. The tasks are either replicas of or analogous to the kinds of
problems faced by adult citizens and consumers or professionals in the
field." -- Grant Wiggins -- (Wiggins 1993, p229).
"Performance assessments call upon the examinee to demonstrate specific
skills and competencies, that is, to apply the skills and knowledge they have
mastered." -- Richard J. Stiggins -- (Stiggins, 1987, p. 34).
Konstantinos Mathiopoulos
Fotini Paraskeva
4
Department of Technology Education and Digital Systems
University of Piraeus
Traditional Assessment VS Authentic Assessment
Multiple choice or other tests that are
easily marked, often revealing only
whether students can recognise, recall or
“plug in” what was learned out of context.
Linking learning and working by
creating a correspondence between
what is assessed in school and what
students need to do in the workplace.
Konstantinos Mathiopoulos
Fotini Paraskeva
5
Department of Technology Education and Digital Systems
University of Piraeus
Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, from Park University
Konstantinos Mathiopoulos
Fotini Paraskeva
6
Department of Technology Education and Digital Systems
University of Piraeus
Authentic Assessments in general are expected to:
• stimulate students to learn more deeply
(Birenbaum, 1996; Dochy & McDowell, 1997; McDowell, 1995; Frederiksen, 1984)
•stimulate students to develop professionally relevant
skills and thinking processes used by professionals
(Gielen et al; Savery & Duffy, 1995)
•motivate students to learn by showing the immediate
relevance of that which is learnt for professional practice
(Herrington & Herrington, 1998; Lizzio & Wilson, 2004a; Martens, Gulikers, &
Bastiaens, 2004; McDowell)
Konstantinos Mathiopoulos
Fotini Paraskeva
7
Department of Technology Education and Digital Systems
University of Piraeus
Research gap:
New ways of assessment are required so as to replace
standardized tests and quizzes.
It is important to examine how authentic assessment
could be integrated in a web based environment in
higher education.
The impact of authentic assessment on student
learning in a computer supported collaborative learning
environment is an open hot issue in higher education.
Konstantinos Mathiopoulos
Fotini Paraskeva
8
Department of Technology Education and Digital Systems
University of Piraeus
Goal of the paper
This paper is a report on findings of a study focused on
examining perceptions of assessment authenticity and the
influence on student learning in a computer supported
collaborative learning environment at higher education
level.
A growing body of literature and research on new modes
of assessment stresses that the effects of assessments
on student learning should always be examined in the
light of the whole learning environment along with student
perception of the learning environment (Biggs, 1996;
Birenbaum, 1996; Segers, Dierick, & Dochy, 2001;
Struyven, 2005).
Konstantinos Mathiopoulos
Fotini Paraskeva
9
Department of Technology Education and Digital Systems
University of Piraeus
The method: Research Questions
• Does authentic assessment change the way that
students think about assessment in general?
• Does implementation of authentic assessment
improve study approach(deep/surface)?
• Does authentic
thinking?
assessment
promote
reflective
• Does authentic assessment increase motivation of
learners in a computer supported collaborative
learning environment?
Konstantinos Mathiopoulos
Fotini Paraskeva
10
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Participants
Participants were thirty-six post graduate students
who attend a two-year Master's Degree (M.Sc.) in a
department of "Information and Digital Systems"
(120 ECTS) in the area of e-Learning. Students had
various educational and professional backgrounds
and age range from 22 to 55 years. Data was
collected during the course of Computer Mediated
Communication and Collaboration.
Konstantinos Mathiopoulos
Fotini Paraskeva
11
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Instruments
The moodle course management system was used (http://eptlab.unipi.gr),
in order for students to be able to communicate through chat and forums,
access documents and presentations, participate in workshops (project
submission, self-assessment and peer assessment) and answer to
questionnaires.
Konstantinos Mathiopoulos
Fotini Paraskeva
12
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Instruments
A 41- item questionnaire for perception of authenticity based on
5-dimensionsal framework for authenticity (Gulikers, Bastiaens, &
Kirschner, 2004; 2006) and on perceptions of the purpose of formative
computer based assessment through quiz items (Sandra T. Miller, 2009)
•
The 20-item deep and surface study approach questionnaire based on
Revised-Study Process Questionnaire - 2 Factors (R-SPQ-2F; Biggs,
Kember, & Leung, 2002).
The 16-item reflective thinking questionnaire based on the Level of
Reflective Thinking Questionnaire (Kember D. et al, 2000).
The 81-item motives, cognitive and meta-cognitive strategies
questionnaire based on the Motivated Strategies for Learning
Questionnaire (Pintrich, R. R., & DeGroot, E. V. (1990).
Konstantinos Mathiopoulos
Fotini Paraskeva
13
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
Pearson r Correlation
Surface
Deep Study
Approach
Study
Overall
Approach
Authenticity
Task
Physical
Form/
Context
Result
Criteria
Criteria
Perception of
Transparency
Purpose
Overall Authenticity
*
,390
-,176
Task
,410*
-,244
,834**
Physical Context
,297
-,175
,633**
,680**
Form /Result
,364*
-,255
,745**
,798**
,727**
Criteria
,297
-,324
,768**
,777**
,751**
,833**
Criteria Transparency
,348*
-,157
,579**
,544**
,656**
,672**
,660**
Perception of
,401*
-,339*
,737**
,722**
,709**
,766**
,680**
,570**
,395*
-,524**
,699**
,675**
,761**
,796**
,725**
,580**
(Low Cronbach a)
Purpose
Value of AA
,795**
Note: * Correlation is significant at the 0.05 level (2-tailed).
** Correlation is significant at the 0.01 level (2-tailed).
Sample size: N=36
Table1: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and
study approach.
Konstantinos Mathiopoulos
Fotini Paraskeva
14
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
Pearson r correlation analysis indicates that authentic assessment
indicators have positive linear relationship with deep study approach.
Statistically significant is the correlation between deep study approach and
overall authenticity r(36) = .390 p< .05, Form /Result r(36) = .364 p< .05,
Criteria Transparency r(36) = .348 p< .05, Perception Of Purpose r(36)= .
401 p< .05 and Value of Authentic Assessment r(36)= .395 p< .05.
On the contrary, authentic assessment indicators have negative linear
correlation with surface study approach.
Statistically significant factors are Perception Of Purpose r(36)= - .339 p< .
05 and Value of Authentic Assessment r(36)= - .524 p< .01.
Konstantinos Mathiopoulos
Fotini Paraskeva
15
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
There is positive linear correlation between Perception of Purpose of
Authentic Assessment and the dimensions of authentic assessment.
Overall authenticity r(36)=.737 p< .01, Physical Context r(36)=.709 p< .01,
Form/Result r(36)=.766 p< .01, Criteria r(36)=.680 p< .01, Criteria
Transparency r(36)=. 570 p< .01 apart from task factor due to low internal
consistency Cronbach a=.436.
There is positive linear correlation between Value of Authentic
Assessment and the dimensions of authentic assessment.
Overall authenticity r(36)=.699 p< .01, Physical Context r(36)=.761 p
University of Piraeus
Authentic Assessment in Computer
Supported Collaborative Learning
Environments in Higher Education
Konstantinos Mathiopoulos
Fotini Paraskeva
University of Piraeus, Department of Technology Education and
Digital Systems
Konstantinos Mathiopoulos
Fotini Paraskeva
1
Department of Technology Education and Digital Systems
University of Piraeus
Contents at a glance
•
•
•
•
•
•
•
•
Theoretical background of Authentic Assessment
Definitions of Authentic Assessment
Traditional Assessment VS Authentic Assessment
Advantages and disadvantages of Authentic Assessment
Authentic Assessment’s expectations
Research gap
Goal of the paper
Study on Authentic Assessment in Computer Supported
Collaborative Learning Environments in Higher Education
• Conclusions
• Future research
Konstantinos Mathiopoulos
Fotini Paraskeva
2
Department of Technology Education and Digital Systems
University of Piraeus
Konstantinos Mathiopoulos
Fotini Paraskeva
3
Department of Technology Education and Digital Systems
University of Piraeus
Definitions of Authentic Assessment:
"A form of assessment in which students are asked to perform real-world
tasks that demonstrate meaningful application of essential knowledge and
skills "-- Jon Mueller 2008
"...Engaging and worthy problems or questions of importance, in which
students must use knowledge to fashion performances effectively and
creatively. The tasks are either replicas of or analogous to the kinds of
problems faced by adult citizens and consumers or professionals in the
field." -- Grant Wiggins -- (Wiggins 1993, p229).
"Performance assessments call upon the examinee to demonstrate specific
skills and competencies, that is, to apply the skills and knowledge they have
mastered." -- Richard J. Stiggins -- (Stiggins, 1987, p. 34).
Konstantinos Mathiopoulos
Fotini Paraskeva
4
Department of Technology Education and Digital Systems
University of Piraeus
Traditional Assessment VS Authentic Assessment
Multiple choice or other tests that are
easily marked, often revealing only
whether students can recognise, recall or
“plug in” what was learned out of context.
Linking learning and working by
creating a correspondence between
what is assessed in school and what
students need to do in the workplace.
Konstantinos Mathiopoulos
Fotini Paraskeva
5
Department of Technology Education and Digital Systems
University of Piraeus
Mandernach, B. J. (2003). Incorporating Authentic Assessment. Retrieved 01-05-09, from Park University
Konstantinos Mathiopoulos
Fotini Paraskeva
6
Department of Technology Education and Digital Systems
University of Piraeus
Authentic Assessments in general are expected to:
• stimulate students to learn more deeply
(Birenbaum, 1996; Dochy & McDowell, 1997; McDowell, 1995; Frederiksen, 1984)
•stimulate students to develop professionally relevant
skills and thinking processes used by professionals
(Gielen et al; Savery & Duffy, 1995)
•motivate students to learn by showing the immediate
relevance of that which is learnt for professional practice
(Herrington & Herrington, 1998; Lizzio & Wilson, 2004a; Martens, Gulikers, &
Bastiaens, 2004; McDowell)
Konstantinos Mathiopoulos
Fotini Paraskeva
7
Department of Technology Education and Digital Systems
University of Piraeus
Research gap:
New ways of assessment are required so as to replace
standardized tests and quizzes.
It is important to examine how authentic assessment
could be integrated in a web based environment in
higher education.
The impact of authentic assessment on student
learning in a computer supported collaborative learning
environment is an open hot issue in higher education.
Konstantinos Mathiopoulos
Fotini Paraskeva
8
Department of Technology Education and Digital Systems
University of Piraeus
Goal of the paper
This paper is a report on findings of a study focused on
examining perceptions of assessment authenticity and the
influence on student learning in a computer supported
collaborative learning environment at higher education
level.
A growing body of literature and research on new modes
of assessment stresses that the effects of assessments
on student learning should always be examined in the
light of the whole learning environment along with student
perception of the learning environment (Biggs, 1996;
Birenbaum, 1996; Segers, Dierick, & Dochy, 2001;
Struyven, 2005).
Konstantinos Mathiopoulos
Fotini Paraskeva
9
Department of Technology Education and Digital Systems
University of Piraeus
The method: Research Questions
• Does authentic assessment change the way that
students think about assessment in general?
• Does implementation of authentic assessment
improve study approach(deep/surface)?
• Does authentic
thinking?
assessment
promote
reflective
• Does authentic assessment increase motivation of
learners in a computer supported collaborative
learning environment?
Konstantinos Mathiopoulos
Fotini Paraskeva
10
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Participants
Participants were thirty-six post graduate students
who attend a two-year Master's Degree (M.Sc.) in a
department of "Information and Digital Systems"
(120 ECTS) in the area of e-Learning. Students had
various educational and professional backgrounds
and age range from 22 to 55 years. Data was
collected during the course of Computer Mediated
Communication and Collaboration.
Konstantinos Mathiopoulos
Fotini Paraskeva
11
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Instruments
The moodle course management system was used (http://eptlab.unipi.gr),
in order for students to be able to communicate through chat and forums,
access documents and presentations, participate in workshops (project
submission, self-assessment and peer assessment) and answer to
questionnaires.
Konstantinos Mathiopoulos
Fotini Paraskeva
12
Department of Technology Education and Digital Systems
University of Piraeus
The Method: The Instruments
A 41- item questionnaire for perception of authenticity based on
5-dimensionsal framework for authenticity (Gulikers, Bastiaens, &
Kirschner, 2004; 2006) and on perceptions of the purpose of formative
computer based assessment through quiz items (Sandra T. Miller, 2009)
•
The 20-item deep and surface study approach questionnaire based on
Revised-Study Process Questionnaire - 2 Factors (R-SPQ-2F; Biggs,
Kember, & Leung, 2002).
The 16-item reflective thinking questionnaire based on the Level of
Reflective Thinking Questionnaire (Kember D. et al, 2000).
The 81-item motives, cognitive and meta-cognitive strategies
questionnaire based on the Motivated Strategies for Learning
Questionnaire (Pintrich, R. R., & DeGroot, E. V. (1990).
Konstantinos Mathiopoulos
Fotini Paraskeva
13
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
Pearson r Correlation
Surface
Deep Study
Approach
Study
Overall
Approach
Authenticity
Task
Physical
Form/
Context
Result
Criteria
Criteria
Perception of
Transparency
Purpose
Overall Authenticity
*
,390
-,176
Task
,410*
-,244
,834**
Physical Context
,297
-,175
,633**
,680**
Form /Result
,364*
-,255
,745**
,798**
,727**
Criteria
,297
-,324
,768**
,777**
,751**
,833**
Criteria Transparency
,348*
-,157
,579**
,544**
,656**
,672**
,660**
Perception of
,401*
-,339*
,737**
,722**
,709**
,766**
,680**
,570**
,395*
-,524**
,699**
,675**
,761**
,796**
,725**
,580**
(Low Cronbach a)
Purpose
Value of AA
,795**
Note: * Correlation is significant at the 0.05 level (2-tailed).
** Correlation is significant at the 0.01 level (2-tailed).
Sample size: N=36
Table1: Pearson r correlation analysis of dimensions of authentic assessment, perceptions of authentic assessment and
study approach.
Konstantinos Mathiopoulos
Fotini Paraskeva
14
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
Pearson r correlation analysis indicates that authentic assessment
indicators have positive linear relationship with deep study approach.
Statistically significant is the correlation between deep study approach and
overall authenticity r(36) = .390 p< .05, Form /Result r(36) = .364 p< .05,
Criteria Transparency r(36) = .348 p< .05, Perception Of Purpose r(36)= .
401 p< .05 and Value of Authentic Assessment r(36)= .395 p< .05.
On the contrary, authentic assessment indicators have negative linear
correlation with surface study approach.
Statistically significant factors are Perception Of Purpose r(36)= - .339 p< .
05 and Value of Authentic Assessment r(36)= - .524 p< .01.
Konstantinos Mathiopoulos
Fotini Paraskeva
15
Department of Technology Education and Digital Systems
University of Piraeus
The Statistical Results of the Research
There is positive linear correlation between Perception of Purpose of
Authentic Assessment and the dimensions of authentic assessment.
Overall authenticity r(36)=.737 p< .01, Physical Context r(36)=.709 p< .01,
Form/Result r(36)=.766 p< .01, Criteria r(36)=.680 p< .01, Criteria
Transparency r(36)=. 570 p< .01 apart from task factor due to low internal
consistency Cronbach a=.436.
There is positive linear correlation between Value of Authentic
Assessment and the dimensions of authentic assessment.
Overall authenticity r(36)=.699 p< .01, Physical Context r(36)=.761 p