How to develop the system
Workshop dan Seminar ICT ASEAN Human
Development Program.
JICA-IIAI Japan dengan Universitas Kristen Duta Wacana
15, September 2017, 10:45AM-12:00PM.
How to Develop Foreign Language
Education Support System
Yuichi Ono, Ph.D
Faculty of Humanities and Social Sciences
Center for Education of Global Communication(CEGLOC)
University of Tsukuba, Japan
ono.yuichi.ga@u.tsukuba.ac.jp
Table of Contents
nIntroduction
nSystem Development
nSystem Description
nExperimental Study for System Evaluation
nConclusion
• Let’s have a review of the relationship
between technology and pedagogy.
Technology ✕ Pedagogical Principles ⇒ Paradigm
S
c a CALL
C
CALL
ica i e
Ce
y
I eg a i e CALL
Technology Mainframe
PCs
Multimedia and
Internet
Englishteaching
program
GrammarTranslation and
Audio Lingual
Communicate
[sic]language
teaching
Content-Based,
ESP/EAP
View of
Language
Structural
(a formal structural
system)
Cognitive
(a mentally
constructed
system)
Socio-cognitive
(developed in
social interaction)
Principal
use of
computers
Drill and practice
Communicative
exercises
Authentic
discourse
Principal
objective
Accuracy
And fluency
And agency
Mark Warschauer and Deborah Healey: “Computers and language learning: an overview”,
Language Teaching, Vol. 31, pp. 57–71 (1998) ʤPartially modified by the authorʥ
Recent trends in Japan
• In Japan, the Ministry of Education, Culture, Sports,
Science and Technology (MEXT) is promoting the use
of ICT in education.
• Furthermore, MEXT in Japan has a clear agenda that
tablets will be incorporated into the classroom in all
schools by 2020 to achieve the goal of having each
student in Japan use the tablet one by one in class.
• MEXT also promoting digital-textbook (eBooks)
introduction along with paper-based textbook.
Ministry of Education, Culture, Sports, Science and Technology in Japan.
2015. http://www.mext.go.jp/english/
Introduction (1/3)
• Universities and colleges all over the world are
advancing the use of Massive Open Online
Courses (MOOC) and Online Educational
Resources (OER) (Allen & Seaman, 2014).
Introduction (2/3)
• MOOC has become a standardized educational
resource. On the other hand, the use of OER has not
spread widely so far. (Shigeta et. al., 2016)
• It is pointed out that only half of the higher
education institutions seem to have been working on
the active use of OER in the US in 2015 (Allen &
Seaman, 2014).
Reasons
n Difficulty in constructing curriculum from OER
n Not controlled for English as a Foreign Language
(EFL) Learners
Introduction (3/3)
• OER materials are the direct instructional resources
for familiarizing students with authentic language use.
• The system for encouraging EFL learners to expand
their interest may lead to “creative” and
“independent” learners.
Creating Truly Motivated and
Independent Learners of English
nLearning to Learn in Practice
“Good schools and good teachers make a difference. A
really effective teacher can add significant value to a
student’s life chances.”
Smith, A., Lovatt, M. and Turner, J. (2009) Learning to Learn in Practice: The L2
Approach. Crown House Publishing.
nPlacing Students at the Heart of Creative Learning
“Learning engages young people in serious, meaningful,
relevant, imaginative and challenging activities and
tasks.”
Owen, N. (2012) Placing Students at the Heart of Creative Learning (Creative
Teaching/Creative Schools). Routledge.
Backgrounds
Educational Technology
●Open Education
●OER
●Learning Analytics
●Personalized Learning
Information Technology
●Web Mining
● Suggestion of Reuse of OER
●Data Mining &
Visualization
● Visualization and Organization of Learning
Portfolio & Learning Behavior Data
●Adaptive Learning
● Feedback of Learning Data and Adaptive
Quiz Generation
システムの紹介：動画から問題を自動生成する
YouTUbe / TedTalk
URL
kamirazio.com/yt/
Automatic Quiz Generation
システムの紹介：リスニングゲームの設定
好きな名前を
コピーしたURL
ゲームのモードを選ぶ
・穴埋めゲーム
・タッチゲーム（並べ替え）
・タイピングゲーム
穴埋めのクイズの量
(ブランクゲームのみ)
チャンク: 文の区切りが短くなる
ゲームのタスクカードが作成される
システムの紹介：ゲームの管理
達成量
・モード
・問題量
・翻訳言語
・映像元
・点数
スタートボタン
コピー/ 問題編集 /クラスルーム追加 / 削除
システムの紹介：ゲームのインターフェース
Enter
Repeat
Space
Next stage
▶
Hint
◀
Skip (Three
Mistakes)
▼(Click)
Dictionary
• The words that need to be reviewed are listed in various sizes, that is, the larger
words represent those the user needs to check with more care.
• The blue font : a dictionary use
• The orange : mistakes made.
• The number: the ranks specified by JACET 8000 vocabulary list
(http://language.sakura.ne.jp/s/voc.html).
• The vertical axis: the number of repeated trials.
• The horizontal axis: the difficulty rank provided by JACET 8000.
Feedback for Reflection
Pilot Study
nFlipped Classroom model is becoming more
popular in Japan under the Free Movement
(Nakajima & Ono, 2015).
nHow will the OER learning support system
supports Japanese EFL learners for “creative”
learning depending on their interest?
System Development
• Automatic Quiz Generator “TED Note”
• Updated Version “YouTutors”
Points of modification
(i) a more user-friendly interface
(ii) Learning Management System (LMS) functions
like class management
(iii) more factors for gamification
(iv) visualization of learning behavior
Study
(RQ1) Does the combined use of OER and the
proposed system have any effect on Japanese EFL
learners’ motivation?
• Pre- and post- paired t-test on the Motivational
Survey
• Factor Analysis of Post-test results to produce
interrelationship among factors
(RQ2) How did the learners use the system?
• Semi-structured interview of four randomly chosen
participants
RQ1: Motivational Survey
• ARCS Model (Keller, 2010)
• The ARCS model is rooted in the idea that learner’s
motivation can be analyzed as consisting of four
independent aspects: Attention, Relevance, Confidence,
and Satisfaction.
• If a student learns successfully, the arisen motivation at
the first stage of learning (Attention) is followed by the
feeling that this content is relevant to their lives or
goals (Relevance). If the learning process makes them
realize that the learning will be successful if they try
(Confidence), then it ends with Satisfaction, leading to
them paying Attention in the next class.
RQ1: Instrument
• The questionnaire scale: Instructional Materials
Motivational Survey (IMMS) (Huang & Yoo, 2010;
Keller, 2010)
• Consisting of 36 items, divided into four basic
components (Attention, Relevance, Confidence,
and Satisfaction).
• We adopted a 5-point Likert Scale for the individual
questionnaire items.
A
There was something interesting at the beginning of YouTutors
that got my attention.
C
After working on YouTutors for a while, I was confident that I
would be able to pass a test on.
A
The variety of reading passages, exercises, illustrations, etc.,
helped keep my attention on the tutorial.
C
I could not really understand quite a bit of the material in
YouTutors.
A
The style of writing is boring.
C
The good organization of the content helped me be confident
that I would learn this material.
A
There are so many words on each window that it is irritating.
R
It is clear to me how the content of YouTutors is related to
things I already know.
A
YouTutors is eye-catching.
R
There were examples that showed me how YouTutors could be
important to some people.
A
The quality of the game helped to hold my attention.
R
Completing YouTutors successfully was important to me.
A
YouTutors is so abstract that it was hard to keep my attention on
it.
R
The content of YouTutors is relevant to my interests.
A
The design of YouTutors looks dry and unappealing.
R
The content and style of writing in YouTutors convey the
impression that its content is worth knowing.
A
The way the information is arranged in YouTutors helped keep
my attention.
R
YouTutors was not relevant to my needs because I already knew
most of it.
A
YouTutors has things that stimulated my curiosity.
R
I could relate the content of YouTutors to things I have seen,
done or thought about in my own life.
A
The amount of repetition in YouTutors caused me to get bored
sometimes.
R
The content YouTutors will be useful to me.
A
I learned some things that were surprising or unexpected.
S
Completing the exercises in YouTutors gave me a satisfying
feeling of accomplishment.
C
When I first looked at YouTutors, I had the impression that it
would be easy for me.
S
I enjoyed YouTutors so much that I would like to know more
about his topic.
C
YouTutors was more difficult to understand than I would like for
it to be.
S
I really enjoyed studying with YouTutors.
C
After reading the introductory information, I felt confident that I
knew what I was supposed to learn from YouTutors.
S
The working of feedback after the exercises, or of other
comments in YouTutors, helped me feel rewarded for my effort.
C
Many of the pages had so much information that it was hard to
pick out and remember the important points.
S
It felt good to successfully complete YouTutors.
C
As I worked on YouTutors, I was confident that I could learn the
content.
S
It was a pleasure to work on such a well-designed system.
C
The exercises in YouTutors were too difficult.
S
I would like to use YouTutors for my self-study from now.
RQ1: Results
• A total of 72 first year students participated in the
two-week experiment in an EGAP course. (α=.905)
• Results of paired t-test showed that almost all of
the question (30/36) items showed significant
differences.
• Maximum Likelihood Method of Promax Rotation
produced four factors with 15 items were extracted.
Descriptive Statistics
Question Item
M
SD
Skew
Rotated Factor Pattern
Kurt
Factor1
Factor2
Factor3
Factor4
S
I really enjoyed studying with YouTutors.
3.99
0.87
-0.45
-0.56
0.914
-0.078
-0.050
0.147
S
The working of feedback after the exercises, or of other comments in
3.56
YouTutors, helped me feel rewarded for my effort.
0.93
-0.19
-0.28
0.862
-0.253
-0.058
0.116
S
I would like to use YouTutors for my self-study from now.
3.88
0.90
-0.67
0.45
0.701
0.030
0.026
0.206
A
The variety of reading passages, exercises, illustrations, etc., helped
4.07
keep my attention on the tutorial.
0.84
-0.53
-0.41
0.631
0.385
-0.088
-0.221
S
I enjoyed YouTutors so much that I would like to know more about
3.75
his topic.
There was something interesting at the beginning of YouTutors that
4.05
got my attention.
0.89
-0.18
-0.72
0.612
0.210
0.135
0.013
0.85
-0.53
-0.41
0.508
0.407
0.160
-0.225
A
R
The content YouTutors will be useful to me.
3.77
0.77
-0.83
1.62
0.501
0.024
0.197
0.251
A
The quality of the game helped to hold my attention.
3.78
0.84
-0.16
-0.58
-0.073
0.819
-0.162
0.188
A
YouTutors has things that stimulated my curiosity.
3.81
0.84
-0.58
0.69
0.216
0.691
-0.179
0.129
A
I learned some things that were surprising or unexpected.
2.99
1.02
-0.08
-0.27
-0.071
0.586
0.085
0.014
R
The content and style of writing in YouTutors convey the impression
that its content is worth knowing.
There were examples that showed me how YouTutors could be
important to some people.
It is clear to me how the content of YouTutors is related to things I
already know.
The way the information is arranged in YouTutors helped keep my
attention.
The good organization of the content helped me be confident that I
would learn this material.
3.68
0.91
-0.54
0.10
0.081
-0.101
0.692
-0.213
2.38
1.09
0.46
-0.57
0.014
-0.172
0.594
-0.021
2.63
0.95
0.24
-0.17
-0.125
0.232
0.496
0.046
3.37
0.89
-0.23
-0.29
0.206
0.085
-0.228
0.492
2.99
0.81
-0.14
0.70
-0.066
0.331
0.355
0.423
0.422
0.492
0.540
0.572
R
R
A
C
Accumulated Factor Loadings
Correlation among Factors
Factor2
0.733
Factor3
0.365
0.390
Factor4
0.357
0.412
0.354
RQ2: Semi-Structured Interview
nFour first year students volunteered to carry out selfstudy with YouTutors independent of the course.
nTheir requirement was that they had to work on at
least one Ted Talk video of their interest every day
for a week, which means that they were expected to
complete at least seven videos in a week.
nIn order to investigate how they worked on the
system during the week, we carried out a semistructured interview and asked the following
questions in Japanese along with their reflection on
the visualized learning process data. We have
translated them into English in this paper.
RQ2: Semi-Structured Interview
1)Is it difficult to continue the task for a week?
2)When and how long did you work on YouTutors in
a day?
3)What did you enjoy when studying?
4)What was painful when studying?
5)What was the standard of your choice?
6)What is important in continuing your studies?
7)Others
Video Titles Participants Chose
Participants A
Participants B
Are you human? / Ze Frank
1,000 TED Talks in six words / Sebastian Wernicke
The world's English mania / Jay Walker
The year open data went worldwide / Tim Berners-Lee
Selling condoms in the Congo / Amy Lockwood
The quantified self / Gary Wolf
The surprising way groups like ISIS stay in power /
Benedetta Berti
Physical therapy is boring -- play a game instead / Cosmin
Mihaiu
How to succeed? Get more sleep / Arianna Huffington
How I harnessed the wind / William Kamkwamba
The lost art of letter-writing / Lakshmi Pratury
The best gift I ever survived / Stacey Kramer
Letʼs try emotional correctness / Sally Kohn
A story of mixed emoticons / Rives
How to use a paper towel / Joe Smith
Participants C
Participants D
How Syria's architecture laid the foundation for brutal war / Looks aren't everything. Believe me, I'm a model. / Cameron
Marwa Al-Sabouni
Russell
How I fell in love with quasars, blazars and our incredible
Street art with a message of hope and peace / eL Seed
universe / Jedidah Isler
Wry photos that turn stereotypes upside down / Uldus
What's so funny about mental illness? / Ruby Wax
Bakhtiozina
What I've learned from my autistic brothers / Faith Jegede
It's TED, the Musical / TED staff
Cole
For these women, reading is a daring act / Laura Boushnak "High School Training Ground" / Malcolm London
Poems of war, peace, women, power / Suheir Hammad
Two poems about what dogs think (probably) / Billy Collins
How to raise a black son in America / Clint Smith
Why is 'x' the unknown? / Terry Moore
Questions
Is it difficult to keep
the task in a week?
Participant A
Not so much.
Participant B
Participant C
Participant D
Very hard. Tempted to give up.
Didn’t have a computer so they I was busy, so it was difficult to
have to go to satellite. Thus very have time for the study. The
troublesome but enjoyed.
study is enjoyable.
When and how long About 20 minutes at night.
did you work on
YouTutors in a
day?
At night after club activity.
More than an hour when free,
but less than 20 minutes when
tired.
After club activity, went to
satellite, and tried about an
hour.
At night, before going to bed.
30-60 minutes a day.
What did you enjoy
during the study?
The contents of Ted were
interesting. Chose the
video I am interested in.
Enjoyed choosing the video.
Found some funny videos. Also
enjoyed Japanese videos.
Enjoyed an easy video, slowly
spoken. Enjoyed following the
stories.
What was painful
during the study?
Some difficuly
vocabularies. But did not
use hint-module.
Exhaused with some words I
know but whose spelling I did
not know. But little by little, I
came to like typing mode.
Some speeches were difficult to
understand and exhausting.
Enjoued observing various kinds
of videos. When I got the right
answer at the first try, I was
really excited.
I was exausted when I had
difficulty in listening and when I
faced with unkown words.
What was the
standard of your
choice?
Chose the movies of my
interest. I was interested
in social issues.
Chose less academ,ic, informal
and friendly movies. I love
“Paper Towel” movie. Chose a
Japanese movie for translation
practice.
Chose the movie of my interest
at first. But chose movies about
“Statistics” later.
Chose short movies. Want to
choose my own movie, not
assigned by the instructor.
Need some reinforcement by the
instructor to a certain degree.
Decided to buy a computer.
No idea.
Want a reward when I reach a
certain level.
I like to study geography.I like
listening. I want to study
tourism. I want to improve
practical speaking skill.
I like programming. I want to
work internationally. TED
speakiers are good models for
me. I am not good at listening.
What is important to Feedback was interesting.
keep your study?
Others
I like English. I want to go I like English especially I want
abroad to study. I am now to go to Canade for study. Want
working on TOEFL study. to speak English well.
Mistakes
Few
Few
Some
Frequent
Some
Frequent
Some
Frequent
Frequent
Frequent
Some
Some
Repeat
Some
Some
Frequent
Frequent
Dictionary Use
Hint-mode
Summary
• Participant A worked on the task smoothly and seems
highly proficient because of fewer dictionary uses, hint
modes, mistakes, and repeated tries.
• Participants B and D appear to have a lot of difficulty in
completing the task. This is reflected in the frequencies of
the functions used.
• Note that the materials and difficulty levels are not
controlled due to the lack of restriction on the choice of
videos, and it is not certain if the frequency is related to
the psychological condition of the users or their
proficiency levels.
• However, given a sufficiently large number of participants
with a huge amount of “big” data on the learning process,
it might be proved that these frequencies can reflect
learners’ motivation factors to a certain degree.
• Participants seem to have uniquely chosen various videos.
This is naturally expected considering the large number
of OERs on the web.
Summary
• A wider range of choices of OER may have a
possibility to encourage a wider range of users to
pick them up for a wider range of purposes, leading
to an enhancement of their creative, autonomous,
self-regulated, and personalized learning.
Conclusion
• This study confirmed the relationship among four
components in the ARCS Model. In addition, the
data suggested that the choice of contents (OER)
should be carefully chosen by students in terms of
topic and difficulty.
• The integration of OER, its learning support system,
and the instructional design is an important issue
that is worth pursuing in the near future.
Thank you for listening!
Yuichi Ono
Center for Education of Global Communication (CEGLOC),
University of Tsukuba
ono.yuichi.ga@u.tsukuba.ac.jp
Development Program.
JICA-IIAI Japan dengan Universitas Kristen Duta Wacana
15, September 2017, 10:45AM-12:00PM.
How to Develop Foreign Language
Education Support System
Yuichi Ono, Ph.D
Faculty of Humanities and Social Sciences
Center for Education of Global Communication(CEGLOC)
University of Tsukuba, Japan
ono.yuichi.ga@u.tsukuba.ac.jp
Table of Contents
nIntroduction
nSystem Development
nSystem Description
nExperimental Study for System Evaluation
nConclusion
• Let’s have a review of the relationship
between technology and pedagogy.
Technology ✕ Pedagogical Principles ⇒ Paradigm
S
c a CALL
C
CALL
ica i e
Ce
y
I eg a i e CALL
Technology Mainframe
PCs
Multimedia and
Internet
Englishteaching
program
GrammarTranslation and
Audio Lingual
Communicate
[sic]language
teaching
Content-Based,
ESP/EAP
View of
Language
Structural
(a formal structural
system)
Cognitive
(a mentally
constructed
system)
Socio-cognitive
(developed in
social interaction)
Principal
use of
computers
Drill and practice
Communicative
exercises
Authentic
discourse
Principal
objective
Accuracy
And fluency
And agency
Mark Warschauer and Deborah Healey: “Computers and language learning: an overview”,
Language Teaching, Vol. 31, pp. 57–71 (1998) ʤPartially modified by the authorʥ
Recent trends in Japan
• In Japan, the Ministry of Education, Culture, Sports,
Science and Technology (MEXT) is promoting the use
of ICT in education.
• Furthermore, MEXT in Japan has a clear agenda that
tablets will be incorporated into the classroom in all
schools by 2020 to achieve the goal of having each
student in Japan use the tablet one by one in class.
• MEXT also promoting digital-textbook (eBooks)
introduction along with paper-based textbook.
Ministry of Education, Culture, Sports, Science and Technology in Japan.
2015. http://www.mext.go.jp/english/
Introduction (1/3)
• Universities and colleges all over the world are
advancing the use of Massive Open Online
Courses (MOOC) and Online Educational
Resources (OER) (Allen & Seaman, 2014).
Introduction (2/3)
• MOOC has become a standardized educational
resource. On the other hand, the use of OER has not
spread widely so far. (Shigeta et. al., 2016)
• It is pointed out that only half of the higher
education institutions seem to have been working on
the active use of OER in the US in 2015 (Allen &
Seaman, 2014).
Reasons
n Difficulty in constructing curriculum from OER
n Not controlled for English as a Foreign Language
(EFL) Learners
Introduction (3/3)
• OER materials are the direct instructional resources
for familiarizing students with authentic language use.
• The system for encouraging EFL learners to expand
their interest may lead to “creative” and
“independent” learners.
Creating Truly Motivated and
Independent Learners of English
nLearning to Learn in Practice
“Good schools and good teachers make a difference. A
really effective teacher can add significant value to a
student’s life chances.”
Smith, A., Lovatt, M. and Turner, J. (2009) Learning to Learn in Practice: The L2
Approach. Crown House Publishing.
nPlacing Students at the Heart of Creative Learning
“Learning engages young people in serious, meaningful,
relevant, imaginative and challenging activities and
tasks.”
Owen, N. (2012) Placing Students at the Heart of Creative Learning (Creative
Teaching/Creative Schools). Routledge.
Backgrounds
Educational Technology
●Open Education
●OER
●Learning Analytics
●Personalized Learning
Information Technology
●Web Mining
● Suggestion of Reuse of OER
●Data Mining &
Visualization
● Visualization and Organization of Learning
Portfolio & Learning Behavior Data
●Adaptive Learning
● Feedback of Learning Data and Adaptive
Quiz Generation
システムの紹介：動画から問題を自動生成する
YouTUbe / TedTalk
URL
kamirazio.com/yt/
Automatic Quiz Generation
システムの紹介：リスニングゲームの設定
好きな名前を
コピーしたURL
ゲームのモードを選ぶ
・穴埋めゲーム
・タッチゲーム（並べ替え）
・タイピングゲーム
穴埋めのクイズの量
(ブランクゲームのみ)
チャンク: 文の区切りが短くなる
ゲームのタスクカードが作成される
システムの紹介：ゲームの管理
達成量
・モード
・問題量
・翻訳言語
・映像元
・点数
スタートボタン
コピー/ 問題編集 /クラスルーム追加 / 削除
システムの紹介：ゲームのインターフェース
Enter
Repeat
Space
Next stage
▶
Hint
◀
Skip (Three
Mistakes)
▼(Click)
Dictionary
• The words that need to be reviewed are listed in various sizes, that is, the larger
words represent those the user needs to check with more care.
• The blue font : a dictionary use
• The orange : mistakes made.
• The number: the ranks specified by JACET 8000 vocabulary list
(http://language.sakura.ne.jp/s/voc.html).
• The vertical axis: the number of repeated trials.
• The horizontal axis: the difficulty rank provided by JACET 8000.
Feedback for Reflection
Pilot Study
nFlipped Classroom model is becoming more
popular in Japan under the Free Movement
(Nakajima & Ono, 2015).
nHow will the OER learning support system
supports Japanese EFL learners for “creative”
learning depending on their interest?
System Development
• Automatic Quiz Generator “TED Note”
• Updated Version “YouTutors”
Points of modification
(i) a more user-friendly interface
(ii) Learning Management System (LMS) functions
like class management
(iii) more factors for gamification
(iv) visualization of learning behavior
Study
(RQ1) Does the combined use of OER and the
proposed system have any effect on Japanese EFL
learners’ motivation?
• Pre- and post- paired t-test on the Motivational
Survey
• Factor Analysis of Post-test results to produce
interrelationship among factors
(RQ2) How did the learners use the system?
• Semi-structured interview of four randomly chosen
participants
RQ1: Motivational Survey
• ARCS Model (Keller, 2010)
• The ARCS model is rooted in the idea that learner’s
motivation can be analyzed as consisting of four
independent aspects: Attention, Relevance, Confidence,
and Satisfaction.
• If a student learns successfully, the arisen motivation at
the first stage of learning (Attention) is followed by the
feeling that this content is relevant to their lives or
goals (Relevance). If the learning process makes them
realize that the learning will be successful if they try
(Confidence), then it ends with Satisfaction, leading to
them paying Attention in the next class.
RQ1: Instrument
• The questionnaire scale: Instructional Materials
Motivational Survey (IMMS) (Huang & Yoo, 2010;
Keller, 2010)
• Consisting of 36 items, divided into four basic
components (Attention, Relevance, Confidence,
and Satisfaction).
• We adopted a 5-point Likert Scale for the individual
questionnaire items.
A
There was something interesting at the beginning of YouTutors
that got my attention.
C
After working on YouTutors for a while, I was confident that I
would be able to pass a test on.
A
The variety of reading passages, exercises, illustrations, etc.,
helped keep my attention on the tutorial.
C
I could not really understand quite a bit of the material in
YouTutors.
A
The style of writing is boring.
C
The good organization of the content helped me be confident
that I would learn this material.
A
There are so many words on each window that it is irritating.
R
It is clear to me how the content of YouTutors is related to
things I already know.
A
YouTutors is eye-catching.
R
There were examples that showed me how YouTutors could be
important to some people.
A
The quality of the game helped to hold my attention.
R
Completing YouTutors successfully was important to me.
A
YouTutors is so abstract that it was hard to keep my attention on
it.
R
The content of YouTutors is relevant to my interests.
A
The design of YouTutors looks dry and unappealing.
R
The content and style of writing in YouTutors convey the
impression that its content is worth knowing.
A
The way the information is arranged in YouTutors helped keep
my attention.
R
YouTutors was not relevant to my needs because I already knew
most of it.
A
YouTutors has things that stimulated my curiosity.
R
I could relate the content of YouTutors to things I have seen,
done or thought about in my own life.
A
The amount of repetition in YouTutors caused me to get bored
sometimes.
R
The content YouTutors will be useful to me.
A
I learned some things that were surprising or unexpected.
S
Completing the exercises in YouTutors gave me a satisfying
feeling of accomplishment.
C
When I first looked at YouTutors, I had the impression that it
would be easy for me.
S
I enjoyed YouTutors so much that I would like to know more
about his topic.
C
YouTutors was more difficult to understand than I would like for
it to be.
S
I really enjoyed studying with YouTutors.
C
After reading the introductory information, I felt confident that I
knew what I was supposed to learn from YouTutors.
S
The working of feedback after the exercises, or of other
comments in YouTutors, helped me feel rewarded for my effort.
C
Many of the pages had so much information that it was hard to
pick out and remember the important points.
S
It felt good to successfully complete YouTutors.
C
As I worked on YouTutors, I was confident that I could learn the
content.
S
It was a pleasure to work on such a well-designed system.
C
The exercises in YouTutors were too difficult.
S
I would like to use YouTutors for my self-study from now.
RQ1: Results
• A total of 72 first year students participated in the
two-week experiment in an EGAP course. (α=.905)
• Results of paired t-test showed that almost all of
the question (30/36) items showed significant
differences.
• Maximum Likelihood Method of Promax Rotation
produced four factors with 15 items were extracted.
Descriptive Statistics
Question Item
M
SD
Skew
Rotated Factor Pattern
Kurt
Factor1
Factor2
Factor3
Factor4
S
I really enjoyed studying with YouTutors.
3.99
0.87
-0.45
-0.56
0.914
-0.078
-0.050
0.147
S
The working of feedback after the exercises, or of other comments in
3.56
YouTutors, helped me feel rewarded for my effort.
0.93
-0.19
-0.28
0.862
-0.253
-0.058
0.116
S
I would like to use YouTutors for my self-study from now.
3.88
0.90
-0.67
0.45
0.701
0.030
0.026
0.206
A
The variety of reading passages, exercises, illustrations, etc., helped
4.07
keep my attention on the tutorial.
0.84
-0.53
-0.41
0.631
0.385
-0.088
-0.221
S
I enjoyed YouTutors so much that I would like to know more about
3.75
his topic.
There was something interesting at the beginning of YouTutors that
4.05
got my attention.
0.89
-0.18
-0.72
0.612
0.210
0.135
0.013
0.85
-0.53
-0.41
0.508
0.407
0.160
-0.225
A
R
The content YouTutors will be useful to me.
3.77
0.77
-0.83
1.62
0.501
0.024
0.197
0.251
A
The quality of the game helped to hold my attention.
3.78
0.84
-0.16
-0.58
-0.073
0.819
-0.162
0.188
A
YouTutors has things that stimulated my curiosity.
3.81
0.84
-0.58
0.69
0.216
0.691
-0.179
0.129
A
I learned some things that were surprising or unexpected.
2.99
1.02
-0.08
-0.27
-0.071
0.586
0.085
0.014
R
The content and style of writing in YouTutors convey the impression
that its content is worth knowing.
There were examples that showed me how YouTutors could be
important to some people.
It is clear to me how the content of YouTutors is related to things I
already know.
The way the information is arranged in YouTutors helped keep my
attention.
The good organization of the content helped me be confident that I
would learn this material.
3.68
0.91
-0.54
0.10
0.081
-0.101
0.692
-0.213
2.38
1.09
0.46
-0.57
0.014
-0.172
0.594
-0.021
2.63
0.95
0.24
-0.17
-0.125
0.232
0.496
0.046
3.37
0.89
-0.23
-0.29
0.206
0.085
-0.228
0.492
2.99
0.81
-0.14
0.70
-0.066
0.331
0.355
0.423
0.422
0.492
0.540
0.572
R
R
A
C
Accumulated Factor Loadings
Correlation among Factors
Factor2
0.733
Factor3
0.365
0.390
Factor4
0.357
0.412
0.354
RQ2: Semi-Structured Interview
nFour first year students volunteered to carry out selfstudy with YouTutors independent of the course.
nTheir requirement was that they had to work on at
least one Ted Talk video of their interest every day
for a week, which means that they were expected to
complete at least seven videos in a week.
nIn order to investigate how they worked on the
system during the week, we carried out a semistructured interview and asked the following
questions in Japanese along with their reflection on
the visualized learning process data. We have
translated them into English in this paper.
RQ2: Semi-Structured Interview
1)Is it difficult to continue the task for a week?
2)When and how long did you work on YouTutors in
a day?
3)What did you enjoy when studying?
4)What was painful when studying?
5)What was the standard of your choice?
6)What is important in continuing your studies?
7)Others
Video Titles Participants Chose
Participants A
Participants B
Are you human? / Ze Frank
1,000 TED Talks in six words / Sebastian Wernicke
The world's English mania / Jay Walker
The year open data went worldwide / Tim Berners-Lee
Selling condoms in the Congo / Amy Lockwood
The quantified self / Gary Wolf
The surprising way groups like ISIS stay in power /
Benedetta Berti
Physical therapy is boring -- play a game instead / Cosmin
Mihaiu
How to succeed? Get more sleep / Arianna Huffington
How I harnessed the wind / William Kamkwamba
The lost art of letter-writing / Lakshmi Pratury
The best gift I ever survived / Stacey Kramer
Letʼs try emotional correctness / Sally Kohn
A story of mixed emoticons / Rives
How to use a paper towel / Joe Smith
Participants C
Participants D
How Syria's architecture laid the foundation for brutal war / Looks aren't everything. Believe me, I'm a model. / Cameron
Marwa Al-Sabouni
Russell
How I fell in love with quasars, blazars and our incredible
Street art with a message of hope and peace / eL Seed
universe / Jedidah Isler
Wry photos that turn stereotypes upside down / Uldus
What's so funny about mental illness? / Ruby Wax
Bakhtiozina
What I've learned from my autistic brothers / Faith Jegede
It's TED, the Musical / TED staff
Cole
For these women, reading is a daring act / Laura Boushnak "High School Training Ground" / Malcolm London
Poems of war, peace, women, power / Suheir Hammad
Two poems about what dogs think (probably) / Billy Collins
How to raise a black son in America / Clint Smith
Why is 'x' the unknown? / Terry Moore
Questions
Is it difficult to keep
the task in a week?
Participant A
Not so much.
Participant B
Participant C
Participant D
Very hard. Tempted to give up.
Didn’t have a computer so they I was busy, so it was difficult to
have to go to satellite. Thus very have time for the study. The
troublesome but enjoyed.
study is enjoyable.
When and how long About 20 minutes at night.
did you work on
YouTutors in a
day?
At night after club activity.
More than an hour when free,
but less than 20 minutes when
tired.
After club activity, went to
satellite, and tried about an
hour.
At night, before going to bed.
30-60 minutes a day.
What did you enjoy
during the study?
The contents of Ted were
interesting. Chose the
video I am interested in.
Enjoyed choosing the video.
Found some funny videos. Also
enjoyed Japanese videos.
Enjoyed an easy video, slowly
spoken. Enjoyed following the
stories.
What was painful
during the study?
Some difficuly
vocabularies. But did not
use hint-module.
Exhaused with some words I
know but whose spelling I did
not know. But little by little, I
came to like typing mode.
Some speeches were difficult to
understand and exhausting.
Enjoued observing various kinds
of videos. When I got the right
answer at the first try, I was
really excited.
I was exausted when I had
difficulty in listening and when I
faced with unkown words.
What was the
standard of your
choice?
Chose the movies of my
interest. I was interested
in social issues.
Chose less academ,ic, informal
and friendly movies. I love
“Paper Towel” movie. Chose a
Japanese movie for translation
practice.
Chose the movie of my interest
at first. But chose movies about
“Statistics” later.
Chose short movies. Want to
choose my own movie, not
assigned by the instructor.
Need some reinforcement by the
instructor to a certain degree.
Decided to buy a computer.
No idea.
Want a reward when I reach a
certain level.
I like to study geography.I like
listening. I want to study
tourism. I want to improve
practical speaking skill.
I like programming. I want to
work internationally. TED
speakiers are good models for
me. I am not good at listening.
What is important to Feedback was interesting.
keep your study?
Others
I like English. I want to go I like English especially I want
abroad to study. I am now to go to Canade for study. Want
working on TOEFL study. to speak English well.
Mistakes
Few
Few
Some
Frequent
Some
Frequent
Some
Frequent
Frequent
Frequent
Some
Some
Repeat
Some
Some
Frequent
Frequent
Dictionary Use
Hint-mode
Summary
• Participant A worked on the task smoothly and seems
highly proficient because of fewer dictionary uses, hint
modes, mistakes, and repeated tries.
• Participants B and D appear to have a lot of difficulty in
completing the task. This is reflected in the frequencies of
the functions used.
• Note that the materials and difficulty levels are not
controlled due to the lack of restriction on the choice of
videos, and it is not certain if the frequency is related to
the psychological condition of the users or their
proficiency levels.
• However, given a sufficiently large number of participants
with a huge amount of “big” data on the learning process,
it might be proved that these frequencies can reflect
learners’ motivation factors to a certain degree.
• Participants seem to have uniquely chosen various videos.
This is naturally expected considering the large number
of OERs on the web.
Summary
• A wider range of choices of OER may have a
possibility to encourage a wider range of users to
pick them up for a wider range of purposes, leading
to an enhancement of their creative, autonomous,
self-regulated, and personalized learning.
Conclusion
• This study confirmed the relationship among four
components in the ARCS Model. In addition, the
data suggested that the choice of contents (OER)
should be carefully chosen by students in terms of
topic and difficulty.
• The integration of OER, its learning support system,
and the instructional design is an important issue
that is worth pursuing in the near future.
Thank you for listening!
Yuichi Ono
Center for Education of Global Communication (CEGLOC),
University of Tsukuba
ono.yuichi.ga@u.tsukuba.ac.jp