CER-051219.ppt 762KB Jun 23 2011 12:33:20 PM
Computing Education Research
Anders Berglund
Informationsteknologi
Uppsala universitet
Uppsala
anders.berglund@it.uu.se
Informationsteknologi
Computing education research at
Uppsala University
How do our students understand computer
science concepts?
How to teach computer science?
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
This talk
What can we say about how grading in an
internationally distributed project-based course?
and
Why and how to do research in computing
education?
Two parallel
stories
Anders Berglund, Department of Information Technology
My secret agenda:
Inspire to Computing
Education Research
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
What does it mean to learn
something?
Unfortunately (?)….
General
case:
A “meaningless” question
It
all depends on
“what you mean by learning”
or “how you see things”
Anders Berglund, Department of Information Technology
Informationsteknologi
Quantitative/Qualitative research
Quantitative research is grounded on
•
“… the assumption that features of the social environment
constitute an objective reality … collecting numerical data on
observable variables”
Qualitative research is grounded on
•
“… the assumption that individuals construct a social reality
in the form of meanings and interpretations. … studying …
intensively in natural settings”
Implications for the role of the researcher, the concept of
evidence, interpretation etc.
(Gall, Borg & Gall, 1996)
Anders Berglund, Department of Information Technology
Informationsteknologi
A research
approach/methodology/
framework
Offers a way to perform research in
learning
Organizes “ways to see things”
A lens with a certain focus
With a specific research approach:
Some issues get clearer, others blurred
Offers theoretical stand on learning, ways
to see possibilities and limitations, opens
to communicate with other researchers
etc.
Anders Berglund, Department of Information Technology
Research approach/methodology/framework
Phenomenography
Informationsteknologi
(Marton & Booth, 1997)
Takes the learners’ perspective.
Aims at analysing and describing the variation
in students’ experience (understanding,
learning).
A empirical, qualitative research approach
Data is often collected through interviews
Outcome: A few qualitatively different ways, in
which something is understood within a
student cohort
Examples: TCP, experience of being graded
Anders Berglund, Department of Information Technology
Informationsteknologi
Phenomenography
The students
study TCP
T
C
P
The researcher studies
the different ways in
which the students
understand TCP
Researcher
S
tu
d
e
n
tsta
k
in
gaco
u
rse
Anders Berglund, Department of Information Technology
Informationsteknologi
Why the learners’ perspective?
A teacher’s understanding of how her
students understand and learn about
something (CS concepts, for example TCP)
is a good tool for improving teaching.
A change that is not perceived as “good”
by the students does not improve
learning.
Example: Grades are not the driving force for
most students who take (a certain) project
course.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
Background: The Runestone initiative
A project course in computer systems
A project-based course in distributed systems,
real-time programming and computer
communication.
Third/fourth year students majoring in CS.
International collaboration for students who do
not go on exchanges.
Experience of collaboration over ICT tools.
Anders Berglund, Department of Information Technology
Informationsteknologi
Project course in computer systems
The Runestone project
USA
Communication
by e-mail and chat
• 3 + 3 advanced CS students per team
• 16 teams in total
• No lectures
• Tutoring by e-mail and chat
Anders Berglund, Department of Information Technology
Sweden
Informationsteknologi
Student project
Student project: Produce a software system to control a
(modified) Brio labyrinth from any Web-browser.
The task demands computer communication solutions.
The task requires collaboration within the team of 6.
Anders Berglund, Department of Information Technology
Informationsteknologi
Grading in Runestone
Both the process and the project are graded, in
relation to the teams’ own plan
Process grade is based on weekly meetings
Components of grade:
•
•
•
•
Team performance, in relation to the team’s own plan
Individual contribution
Peer evaluation
The instructor’s decision.
Team members are graded by “their” instructor
Different grading schemes in Sweden and US
•
•
Sweden: pass/fail
US: A to E
Anders Berglund, Department of Information Technology
Problematic???
Informationsteknologi
Analysing the grading in Runestone
Teachers’ distribution of grades
(quantitative)
Peer evaluation
The students’ evaluation of each others’
contributions
(quantitative)
Students’ experienced purpose of being graded
(qualitative)
Anders Berglund, Department of Information Technology
Informationsteknologi
Grades
Grades awarded by the instructors, according to
the Runestone scheme (Max = 100, Pass ≈ 60)
To all students
83,61
To Americans
81,55
To Swedes
85,05
Anders Berglund, Department of Information Technology
Informationsteknologi
Peer evaluation
Each student awarded USD 120.- to his
team-mates
From Swede to Swede
22,25
From Swede to American
18,79
From American to American
20,07
From American to Swede
20,07
Then, what is the driving force?
Anders Berglund, Department of Information Technology
Informationsteknologi
The experienced purpose of being
graded
Cat
Getting a
good grade …
Focus is on
1.
… has a value on its
own
The grade per se
2.
… is a tool to reach
other aims
The benefits of a
good grade
3.
… is sub-ordinated to
other aims
Anders Berglund, Department of Information Technology
Me and the team
My team and
other teams
Informationsteknologi
Results on grading
Getting a good grade is not the driving force for
most students in this project.
”Me in the team” or ”My team in front of other
teams” is often important.
How generalizable are these results?
•
Research in computing education research are normally
situationally bound
How can we use this in our teaching?
•
Both ”hard” results and insights gained by doing the
research are useful.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
On Computing Education Research
Theoretically sound research in students’
learning in Computing can serve to improve
teaching.
Different research approaches offer various
contributions to our understanding of students’
learning.
The perspective on ”reality”, what can be
studied, what can be known, what the
researcher’s role is, how research is performed
etc. varies.
This talk: Some examples of
qualitative research
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach
Constructivism
A family of traditions
Jean Piaget, 1896 - 1980
•
•
•
•
Reality is rejected or irrelevant
Knowledge is constructed by each individual
No firm methodology
Passive learning will fail
Extremely influential in school teaching
Anders Berglund, Department of Information Technology
Informationsteknologi
Empirical results from constructivism
in CS Education
Students construct rules for parameters. They
are only sometimes successful. (Fleury, 1991)
Students construct their own understanding of
variables. (Paz, 1996; and others)
Software visualization in itself does not help
students understanding (Mulholland, 1997)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of constructivism in
Computer Science Education
Think twice when using visualizations
Explicitly teach the model of the computer
Don’t start with abstractions
Teach planning, teach to avoid “bricolage”
Don’t run to the computer
Organize “closed labs”
(from Ben-Ari, 2001)
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach:
socio-cultural perspective
A
Thinking/Learning is not influenced by the
environment
⇔
Thinking/Learning is an interaction between the
individual and the environment
A family of traditions in research into learning
Common “source of inspiration”: Lev Vygotsky
(1896 – 1934)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of the soci-cultural
tradition
Why do teams of students interpret a
programming task so differently? (Holland &
Reeves, 1996)
Why do our students hand in “incorrect”
programs? (Ben-David Kolikant, 2005)
The example: Open source community •
Linux
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach:
Critical enquiry
Critical enquiry (research with “a mission”, often
to address power imbalances)
•
Feminist research
Cannot be defined in ontological stand or
research methodology
Instead, it questions fundamental principals and
values and aims to implement changes.
Anders Berglund, Department of Information Technology
Informationsteknologi
Feminist research, an example of
critical research
Are there factors within computer science itself,
that preserves the currently dominating gender
structure? (Björkman & Trojer, 2002)
“We consider it of vital and decisive importance that
gender research is done from within computer science.”
“Such research on the core of computer science and its
knowledge production would serve to enrich computer
science as well as education within computer science.”
Anders Berglund, Department of Information Technology
Informationsteknologi
Selecting a research approach
Why phenomenography?
personal view
Appropriate for the research questions
•
•
•
•
•
•
Results “talk to” computer scientists
Supports deployment of results in teaching
Relevant for recognition of CSE research within CS
Competence at hand/tradition
•
Students’ perspective
A way for students to “talk” to teachers/organisers
Data stems from individuals
Computer science is in focus
•
“Complex” “answers” desirable
Statistical “answers” hard or impossible to get
“Close to” learners
•
A
Shirley Booth
I like it
Anders Berglund, Department of Information Technology
Read!
Create a network!
Informationsteknologi
Literature
Read!
Clancy, M., Stasko, J., Guzdial, M., Fincher, S., & Dale, N. (2001).
Models and Areas for CS Education Research. Computer Science
Education, 11(4), 323-341
Fincher, S., & Petre, M. (2004) Computer Science Education
Research, London, UK: Taylor & Francis
Berglund,
Research
Overview.
Computer
Berglund, A. 2005. Learning computer systems in a distributed
project course: The what, why, how and where. Acta Upsaliensis
Universitatis. Uppsala Dissertations from the Faculty of Science
and Technology 62
A., Daniels, M. and Pears, A. (in press). Qualitative
Projects in Computing Education Research: An
To appear in the Proceedings of the 8th Australasian
Science Education Conference, Hobart, Australia.
Anders Berglund, Department of Information Technology
CeTUSS
Informationsteknologi
CeTUSS ska verka för anpassning av
teknikutbildning till samhällets och
studenternas behov.
Genom att utveckla och sprida
information om lärandemiljöer som är
•
•
•
•
Create a network!
Personligt meningsfulla
Socialt relevanta
Gränsöverskridande
Baserad på samarbete (lokalt och internationellt)
Workshops, kurser etc.
Anders Berglund, Department of Information Technology
Informationsteknologi
Anders Berglund, Department of Information Technology
Anders Berglund
Informationsteknologi
Uppsala universitet
Uppsala
anders.berglund@it.uu.se
Informationsteknologi
Computing education research at
Uppsala University
How do our students understand computer
science concepts?
How to teach computer science?
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
This talk
What can we say about how grading in an
internationally distributed project-based course?
and
Why and how to do research in computing
education?
Two parallel
stories
Anders Berglund, Department of Information Technology
My secret agenda:
Inspire to Computing
Education Research
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
What does it mean to learn
something?
Unfortunately (?)….
General
case:
A “meaningless” question
It
all depends on
“what you mean by learning”
or “how you see things”
Anders Berglund, Department of Information Technology
Informationsteknologi
Quantitative/Qualitative research
Quantitative research is grounded on
•
“… the assumption that features of the social environment
constitute an objective reality … collecting numerical data on
observable variables”
Qualitative research is grounded on
•
“… the assumption that individuals construct a social reality
in the form of meanings and interpretations. … studying …
intensively in natural settings”
Implications for the role of the researcher, the concept of
evidence, interpretation etc.
(Gall, Borg & Gall, 1996)
Anders Berglund, Department of Information Technology
Informationsteknologi
A research
approach/methodology/
framework
Offers a way to perform research in
learning
Organizes “ways to see things”
A lens with a certain focus
With a specific research approach:
Some issues get clearer, others blurred
Offers theoretical stand on learning, ways
to see possibilities and limitations, opens
to communicate with other researchers
etc.
Anders Berglund, Department of Information Technology
Research approach/methodology/framework
Phenomenography
Informationsteknologi
(Marton & Booth, 1997)
Takes the learners’ perspective.
Aims at analysing and describing the variation
in students’ experience (understanding,
learning).
A empirical, qualitative research approach
Data is often collected through interviews
Outcome: A few qualitatively different ways, in
which something is understood within a
student cohort
Examples: TCP, experience of being graded
Anders Berglund, Department of Information Technology
Informationsteknologi
Phenomenography
The students
study TCP
T
C
P
The researcher studies
the different ways in
which the students
understand TCP
Researcher
S
tu
d
e
n
tsta
k
in
gaco
u
rse
Anders Berglund, Department of Information Technology
Informationsteknologi
Why the learners’ perspective?
A teacher’s understanding of how her
students understand and learn about
something (CS concepts, for example TCP)
is a good tool for improving teaching.
A change that is not perceived as “good”
by the students does not improve
learning.
Example: Grades are not the driving force for
most students who take (a certain) project
course.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
Background: The Runestone initiative
A project course in computer systems
A project-based course in distributed systems,
real-time programming and computer
communication.
Third/fourth year students majoring in CS.
International collaboration for students who do
not go on exchanges.
Experience of collaboration over ICT tools.
Anders Berglund, Department of Information Technology
Informationsteknologi
Project course in computer systems
The Runestone project
USA
Communication
by e-mail and chat
• 3 + 3 advanced CS students per team
• 16 teams in total
• No lectures
• Tutoring by e-mail and chat
Anders Berglund, Department of Information Technology
Sweden
Informationsteknologi
Student project
Student project: Produce a software system to control a
(modified) Brio labyrinth from any Web-browser.
The task demands computer communication solutions.
The task requires collaboration within the team of 6.
Anders Berglund, Department of Information Technology
Informationsteknologi
Grading in Runestone
Both the process and the project are graded, in
relation to the teams’ own plan
Process grade is based on weekly meetings
Components of grade:
•
•
•
•
Team performance, in relation to the team’s own plan
Individual contribution
Peer evaluation
The instructor’s decision.
Team members are graded by “their” instructor
Different grading schemes in Sweden and US
•
•
Sweden: pass/fail
US: A to E
Anders Berglund, Department of Information Technology
Problematic???
Informationsteknologi
Analysing the grading in Runestone
Teachers’ distribution of grades
(quantitative)
Peer evaluation
The students’ evaluation of each others’
contributions
(quantitative)
Students’ experienced purpose of being graded
(qualitative)
Anders Berglund, Department of Information Technology
Informationsteknologi
Grades
Grades awarded by the instructors, according to
the Runestone scheme (Max = 100, Pass ≈ 60)
To all students
83,61
To Americans
81,55
To Swedes
85,05
Anders Berglund, Department of Information Technology
Informationsteknologi
Peer evaluation
Each student awarded USD 120.- to his
team-mates
From Swede to Swede
22,25
From Swede to American
18,79
From American to American
20,07
From American to Swede
20,07
Then, what is the driving force?
Anders Berglund, Department of Information Technology
Informationsteknologi
The experienced purpose of being
graded
Cat
Getting a
good grade …
Focus is on
1.
… has a value on its
own
The grade per se
2.
… is a tool to reach
other aims
The benefits of a
good grade
3.
… is sub-ordinated to
other aims
Anders Berglund, Department of Information Technology
Me and the team
My team and
other teams
Informationsteknologi
Results on grading
Getting a good grade is not the driving force for
most students in this project.
”Me in the team” or ”My team in front of other
teams” is often important.
How generalizable are these results?
•
Research in computing education research are normally
situationally bound
How can we use this in our teaching?
•
Both ”hard” results and insights gained by doing the
research are useful.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
On Computing Education Research
Theoretically sound research in students’
learning in Computing can serve to improve
teaching.
Different research approaches offer various
contributions to our understanding of students’
learning.
The perspective on ”reality”, what can be
studied, what can be known, what the
researcher’s role is, how research is performed
etc. varies.
This talk: Some examples of
qualitative research
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach
Constructivism
A family of traditions
Jean Piaget, 1896 - 1980
•
•
•
•
Reality is rejected or irrelevant
Knowledge is constructed by each individual
No firm methodology
Passive learning will fail
Extremely influential in school teaching
Anders Berglund, Department of Information Technology
Informationsteknologi
Empirical results from constructivism
in CS Education
Students construct rules for parameters. They
are only sometimes successful. (Fleury, 1991)
Students construct their own understanding of
variables. (Paz, 1996; and others)
Software visualization in itself does not help
students understanding (Mulholland, 1997)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of constructivism in
Computer Science Education
Think twice when using visualizations
Explicitly teach the model of the computer
Don’t start with abstractions
Teach planning, teach to avoid “bricolage”
Don’t run to the computer
Organize “closed labs”
(from Ben-Ari, 2001)
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach:
socio-cultural perspective
A
Thinking/Learning is not influenced by the
environment
⇔
Thinking/Learning is an interaction between the
individual and the environment
A family of traditions in research into learning
Common “source of inspiration”: Lev Vygotsky
(1896 – 1934)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of the soci-cultural
tradition
Why do teams of students interpret a
programming task so differently? (Holland &
Reeves, 1996)
Why do our students hand in “incorrect”
programs? (Ben-David Kolikant, 2005)
The example: Open source community •
Linux
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach:
Critical enquiry
Critical enquiry (research with “a mission”, often
to address power imbalances)
•
Feminist research
Cannot be defined in ontological stand or
research methodology
Instead, it questions fundamental principals and
values and aims to implement changes.
Anders Berglund, Department of Information Technology
Informationsteknologi
Feminist research, an example of
critical research
Are there factors within computer science itself,
that preserves the currently dominating gender
structure? (Björkman & Trojer, 2002)
“We consider it of vital and decisive importance that
gender research is done from within computer science.”
“Such research on the core of computer science and its
knowledge production would serve to enrich computer
science as well as education within computer science.”
Anders Berglund, Department of Information Technology
Informationsteknologi
Selecting a research approach
Why phenomenography?
personal view
Appropriate for the research questions
•
•
•
•
•
•
Results “talk to” computer scientists
Supports deployment of results in teaching
Relevant for recognition of CSE research within CS
Competence at hand/tradition
•
Students’ perspective
A way for students to “talk” to teachers/organisers
Data stems from individuals
Computer science is in focus
•
“Complex” “answers” desirable
Statistical “answers” hard or impossible to get
“Close to” learners
•
A
Shirley Booth
I like it
Anders Berglund, Department of Information Technology
Read!
Create a network!
Informationsteknologi
Literature
Read!
Clancy, M., Stasko, J., Guzdial, M., Fincher, S., & Dale, N. (2001).
Models and Areas for CS Education Research. Computer Science
Education, 11(4), 323-341
Fincher, S., & Petre, M. (2004) Computer Science Education
Research, London, UK: Taylor & Francis
Berglund,
Research
Overview.
Computer
Berglund, A. 2005. Learning computer systems in a distributed
project course: The what, why, how and where. Acta Upsaliensis
Universitatis. Uppsala Dissertations from the Faculty of Science
and Technology 62
A., Daniels, M. and Pears, A. (in press). Qualitative
Projects in Computing Education Research: An
To appear in the Proceedings of the 8th Australasian
Science Education Conference, Hobart, Australia.
Anders Berglund, Department of Information Technology
CeTUSS
Informationsteknologi
CeTUSS ska verka för anpassning av
teknikutbildning till samhällets och
studenternas behov.
Genom att utveckla och sprida
information om lärandemiljöer som är
•
•
•
•
Create a network!
Personligt meningsfulla
Socialt relevanta
Gränsöverskridande
Baserad på samarbete (lokalt och internationellt)
Workshops, kurser etc.
Anders Berglund, Department of Information Technology
Informationsteknologi
Anders Berglund, Department of Information Technology