T1 112011003 Full text
MULTIMODALITY IN SIMULATION GAMES:
VOCABULARY LEARNING THROUGH VISUAL NOVEL
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Wenda Insani Wangi
112011003
ENGLISH LANGUAGE EDUCATION PROGRAM
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2015
MULTIMODALITY IN SIMULATION GAMES:
VOCABULARY LEARNING THROUGH VISUAL NOVEL
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Wenda Insani Wangi
112011003
ENGLISH LANGUAGE EDUCATION PROGRAM
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2015
i
PUBLICATION AGREEMENT DECLARATION
As a member of the (SWCU) Satya Wacana Christian University academic
community, I verify that:
Name:
Student ID Number:
Study Program:
Faculty:
Kind of Work:
Wenda Insani Wangi
112011003
English Language Education Program
Faculty of Language and Literature
Undergraduate Thesis
In developing my knowledge, I agree to provide SWCU with a non-exclusive
royalty free right for my intellectual property and the contents therein entitled:
Multimodality in Simulation Games: Vocabulary Learning through Visual Novel
along with any pertinent equipment.
With this non-exclusive royalty free right, SWCU maintains the right to copy,
reproduce, print, publish, post, display, incorporate, store in or scan into a
retrieval system or database, transmit, broadcast, barter or sell my intellectual
property, in whole or in part without my express written permission, as long as my
name is still included as the writer.
This declaration is made according to the best of my knowledge.
Made in
Date
: Salatiga
:
Verified by signee,
Wenda Insani Wangi
Approved by
ii
iii
iv
TABLE OF CONTENT
Inner cover ............................................................................................................... i
Approval page ......................................................................................................... ii
Publication agreement declaration ......................................................................... iii
Copyright Statement …………………………………………………………..…iv
ABSTRACT ............................................................................................................ 1
INTRODUCTION................................................................................................... 1
Vocabulary learning ............................................................................................ 2
Advantages of video games in vocabulary acquisition ....................................... 4
Multimodes in visual novel and their role in vocabulary learning ...................... 5
THE STUDY ........................................................................................................... 6
Research Question ............................................................................................... 6
Context of Study .................................................................................................. 6
Participants .......................................................................................................... 7
Instrument of Data Collection ............................................................................. 7
Data Collection Procedure ................................................................................... 7
Data Analysis....................................................................................................... 8
FINDINGS AND DISCUSSION ............................................................................ 8
Visual................................................................................................................... 9
Visual mode with the focus on place. ............................................................ 10
Visual mode with the focus on character‟s expression.. ................................ 10
Visual aid with the focus on event ................................................................. 11
Textual ............................................................................................................... 12
Textual mode with the focus on form. ........................................................... 12
Textual mode with the focus on story. ........................................................... 13
Textual mode with the focus on form and story ............................................ 14
Aural .................................................................................................................. 14
Aural mode with the focus on background music ......................................... 15
Aural mode with the focus on sound effect. .................................................. 15
Multimodal Elements in One Word Learning ................................................... 17
Vocabulary Learning in Visual Novel ............................................................... 19
v
CONCLUSION ..................................................................................................... 20
ACKNOWLEDGEMENT .................................................................................... 22
REFERENCES ...................................................................................................... 22
APPENDIX A ....................................................................................................... 25
APPENDIX B ....................................................................................................... 27
LIST OF TABLE
Table 1: Frequency of Multimodal Modes Used.....................................................9
vi
Multimodality in Simulation Games:
Vocabulary Learning through Visual Novel
Wenda Insani Wangi
ABSTRACT
This research was an analysis to find out the types of multimodes in
simulation game, especially visual novel that can be used in English vocabulary
learning. This research was done due to several claims that learning vocabulary
through video game is effective. The theory that was used to analyze this research
was the theory of Multimodal Learning Environment (Sankey, Birch & Gardiner,
2010), which promote multimodes (visual, aural, and textual) in multimedia as a
tool in learning. The researcher used descriptive approach to find out participants‟
actions in making use of the multimodes in dealing with unknown vocabulary
while playing visual novel, with using Stimulated Recall Protocol method and
interview as the instrument. There were 3 students of batch 2011 from Satya
Wacana Christian University‟s Faculty of Language and Literature as the
participants. It was found that the participants could make use of multimodes in
visual novel in vocabulary learning, although not all of the learning process
resulted in correct meaning. This research may give an insight for the readers to
consider using visual novel as one of multimedia to learn vocabulary and promote
video games as a tool in multimodal learning environment.
Keywords: vocabulary learning, incidental learning, multimodes, visual novel,
simulation game
INTRODUCTION
The claim that video games increase gamers‟ English proficiency has been
spread for a long time, especially in non-native speakers. It is true that people do
not play video games to study, or even do not have any intention to learn while
playing games. Still, the claim stays and constantly declared by gamers, which
provokes pros and cons from society.
1
Regarding to the claim, Meihami, Meihami, & Varmaghani (2013) suggest
that Video games prove potential in promoting language learning and not just for
engaging and entertaining the users. The successfulness of learning vocabulary
through video games might be affected by the existence of multimodes, which is
supported by the theory of multimodal learning (Massaro, 2012). Presentation of
new vocabulary in contexts and multiple exposures to provide clues to meaning
are important for learners to learn new vocabulary. Thus, multimodality in
audiovisual aids which present contextual sounds and illustrative pictures provide
great opportunity for vocabulary learning.
This study was conducted to understand in what way multimodes can be
used by learners in their vocabulary acquisition. The focus of this study was to see
the types of multimodes that learners used in finishing the visual novel every time
they had to deal with unfamiliar words. This paper presents the findings of the
research to show how the participants could make use of multimodes in
vocabulary learning. Moreover, this study would provide more evidence on how
multimodal material could help students in learning vocabulary.
Vocabulary learning
English as a Foreign Language learners usually use various strategies to
memorize vocabulary words, such as word lists (Kim & Gilman, 2008). One of
the ordinary methods in memorizing is learning vocabulary deliberately, which is
also one of the least efficient ways of learning vocabulary (Nation, 2005).
Learning vocabulary through memorizing is quite difficult for students because
they cannot see the real context of each word. DeCarrico (2001) also states that
2
words should not be learnt separately, nor by memorizing without understanding
(cited in Huyen & Nga, 2003).
Thornbury (2002) also suggests that second language learner usually use
their mother tongue as a standard to the target concept, where the new vocabulary
is simply mapped directly onto L1 equivalent. Thus, acquiring vocabulary will be
much more effective if the context which the word takes place can be seen. When
the context can be seen, it is possible to guess the word‟s meaning with the help of
mother tongue, and later learners can use the word for making their own new
sentences. If learners only learn vocabulary by memorizing and do not know how
to use them contextually, then learners only know how to use the vocabulary in an
exact form, but they do not know how to use it in different context in real life
communication. It is because when an exact word is separated from a sentence
and being put in another sentence which has different context, then the word may
change its meaning (related with the study of semantic). That is why both
remembering words and the ability to use them automatically in a wide range of
language contexts become the purpose of vocabulary learning (McCarthy, 1984,
cited in Gu, 2003).
To avoid deliberate vocabulary learning, Krashen (1989) suggests to learn
vocabulary incidentally which is believed to achieve better results than intended
one, which is learned through vocabulary listing - which is out of contexts and not
supported by any aid to help learners guess or see the meaning (cited in Gu,
2003). Nation (2002) also suggests that incidental learning through listening and
reading using context clues can help learners to figure out the meaning of the
3
word. Incidental learning happens when learners „do not realize‟ that they are
currently learning. Here, video games can take position as a medium for incidental
learning.
Advantages of video games in vocabulary acquisition
Nowadays, students are not really fond of reading books, especially when
the book is presented in L2. According to Leu (2002), students who are not fond
of reading books and writing with pencil and paper prefer to learn with the use of
technology. Video games can be one of the technologically-provided media that
can support language acquisition process, which is proved through the study on
the effect of simulation games on sailor and mariners‟ English vocabulary and
pronunciation learning of Khoramshahr Navy University, Iran (deHaan, Reed, and
Kuwada, 2010). The results showed that simulation games had a significant
impact on learners, in which learners who learned English through Navy
Simulator Game English Learning Program performed much better than learners
who only learned English through ordinary method.
The motivational role in vocabulary acquisition process through
simulation game is supported by “self-determination theory: the role of pleasure,
joy, and curiosity in motivating people to do something” (Deci, Vallerand,
Pelletier, and Ryan, 1991). Playing video games usually is done in order to have
fun, so the “burden” to learn is blurred because learners do not realize that they
are currently learning L2 through reading while playing.
The role of video game in vocabulary learning is also supported by Gass &
Selinker (2001) that learning language through simulation video games is
4
contextualized, therefore the learners can acquire language in a natural setting
with various situations which prepare the learners to use the acquired language
properly (cited in Turgut & Irgin, 2009). These contextual aspects are supported
by multimodes in video games.
Multimodes in visual novel and their role in vocabulary learning
Some media, like television, provide instructional elements which are
presented in more than one sensory mode, which may be in the form of visual,
aural, and written – hence, “multimodes” (Sankey, Birch & Gardiner, 2010). It is
believed that the use of multiple representations has been recognized as a very
strong method to smooth the progress of understanding (Moreno, 2002). Kozma
(1991) suggests that multimedia can facilitate learning because the different
advantages of each mode are being put together in one instructional environment.
There is a term called „multimodal learning‟ which supports the theory, where in
the learning situation there are multiple sensory systems which influence learner‟s
actions to understand the unknown words (Massaro, 2012). Mayer (2003) also
argues that combination of words and pictures develops more profound learning,
known as the „multimedia effect‟. In case that an audience cannot understand the
text as the input, the visual representation might aid them to understand the
meaning (Ainsworth & Van Labeke, 2002).
Video games can be included as one of multimedia, and it may facilitate
learning. Although there are lots of game genres, not all video games provide
story plot which is rich of vocabulary for learners. However, genres that provide
story plot and different endings based on the options the audience made during the
5
playtime, engage more in the audience‟s role in incidental learning. That is why
this study used visual novel (which is a sub-genre for simulation game); it is like
reading a novel – rich of vocabularies, but with gaming background. By using
visual novels, learners can focus more in processing vocabulary meaning because
the contexts have been provided by the aids of sounds, texts, and pictures. These
multimodes present additional representations to help learners in understanding
unknown vocabularies. Therefore, visual novels are believed to contribute in
learners‟ L2 acquisition – they provide visualization of the event (pictures), texts
(narratives, dialogues) to observe, and aural (background music, sound effects) to
enhance the vocabulary acquisition.
THE STUDY
Research Question
The study was conducted to answer the following question: “What are the
multimodes in Visual Novel that can be used by learners in understanding
unknown vocabulary?”
Context of Study
This study was a descriptive research which was conducted in January
2015. It was conducted in informal situation. The study had the objective to see
the types of multimodes in visual novel that can be used by learners in learning
English vocabulary. The research used visual novel named “RE: Alistair++” by
Sakevisual, a dating simulation game. The time estimated to finish one route was
around 1 to 2 hours depending on the participants‟ speed in reading and the route
6
itself. With many options and endings the game provided, the chance that
participants encountered different routes was high.
Participants
The researcher chose 3 students of batch 2011 from Satya Wacana
Christian University‟s Faculty of Language and Literature as the participants. The
consideration to take students from batch 2011 was because the last year students
might understand more about reading in English, so the research could focus only
on unfamiliar words. The researcher would use pseudonymes for the participants‟
identity protection.
Instrument of Data Collection
In collecting the data, the researcher used Stimulated Recall Protocol
(SRP) method and interview as the instrument. SRP method is used to investigate
participants‟ thoughts and strategies during doing a task given through asking
participants to report them immediately after they have completed the task, by
presenting tangible visual and aural reminder in the form of video recording of
their performance on the task to help them remember their thought processes
(Gass & Mackey, 2000).
Data Collection Procedure
First, the researcher chose participants from FLL batch 2011 to play the
game from the beginning until the end (approximately 1-2 hours) one by one in
different times. After that, researcher recorded the screen and the audio, and
observed the participant during the playtime. Then, after the participant finished
7
the game, the researcher used SRP method by playing the video recording and
observed it again with the participant. After that, the researcher interviewed the
participant about the multimodes that they used to learn the unfamiliar words
during playing.
Data Analysis
First, the researcher transcribed the interviews into clean transcriptions.
Then, the researcher categorized the modes that were used by the participants in
understanding the unknown vocabulary. The last, the researcher analyzed the data
through the study of Multimodal Learning Environment (Sankey et al, 2010).
FINDINGS AND DISCUSSION
Through the SRP sessions with the three participants – Shin, Yesha, and
Della, the researcher managed to find out how they were able to use the
multimodes in learning English vocabulary from visual novel in details. Based on
Sankey et al‟s (2010) perspective on Multimodal Learning Environments, it was
observed that in “RE: Alistair++” there were 3 instructional elements in different
sensory modes; visual, aural, and written, that were used by the participants in
order to understand the meaning of the unknown words.
8
Table 1: Frequency of multimodes used
Modes
Frequency
%
Visual
14
22
Aural
2
3
Textual
48
75
Total
64
100
As shown in Table 1, the frequency of multimodes used in total was 64 times. The
total number of the unknown words was 51 (see Appendix A). In understanding
some words, each participant used more than one mode, which resulted in the
difference between the total number of the modes used and the total number of the
unknown words. Below is the discussion on how the modes were used by
participants in dealing with the unknown words.
Visual
Visual mode is anything in multimedia which can be seen by the eyes, as
visual sensory perception. In visual novels, usually it can be found in characters‟
physical appearances, characters‟ expressions, situation of place and time, and so
on. Visual mode is the important element that differs „visual novels‟ from the
usual novels, because in visual novel there are visual representations in context
that might help us portraying the scene. Without visual mode, it is just a novel.
9
While playing the game, Shin used the aid of visual mode in 3 unknown
words, Yesha in 9 words, and Della in 2 words (see Appendix A). Furthermore,
the researcher would categorize the visual aid based on three focuses; place,
character‟s expression, and event.
Visual mode with the focus on place. This focus was used when the
details of place was portrayed through image (usually the background picture) and
used by the participants to understand the unknown words. The example was from
Yesha. She said,
“„Chilling‟ di kalimat „just chilling after a long day‟. Si ceweknya itu tanya
„gimana kabarmu‟ gitu, terus cowoknya bilang „not bad‟, „chilling‟ itu mungkin
„refreshing‟ makanya dia jalan-jalan ke mall. Terus ini gambar tempatnya juga di
mall.”
(“‟Chilling in „just chilling after a long day‟; the girl asked „how are you‟ and the
boy said „not bad‟, „chilling‟ here maybe means „refreshing‟ so he went to the
mall. There‟s a picture of the mall as the background.”) From this statement, it
was concluded that Yesha said that because the word „chilling‟ appeared when the
characters were shown being in the mall.
Visual mode with the focus on character’s expression. This focus was
used when the characters‟ expressions were used by the participants to understand
the unknown words. Characters‟ expressions may change depend on the story and
mood, so each character has various expressions. The example was when Shin
learned the meaning of „seethe‟. She said, “Kalo guessing tuh menurutku artinya
„marah‟. Liat dari ekspresinya, sama kalimat sebelumnya kan juga aku
nyinggung-nyinggung keluarganya, terus mungkin dia nggak suka makanya
10
marah.” (“I guessed that it means „angry‟. I looked at his expression and the
sentences before, when I asked about his family. It seemed that he didn‟t like it, so
he became angry.”) Here, Shin used visual mode in learning the word „seethe‟ and
the focus is on character‟s expression.
Visual aid with the focus on event. This focus was used when the
characters were engaged in some actions of doing something that were illustrated
in the picture and used by the participants to learn the unknown words. The
example was when Della learned the word „wafts‟. She said:
“Aku guessing dari kalimatnya, mikir aja sih kan „Steam wafts lazily from the cup,
and I blow gently on it.‟ Terus dari situ udah mikir, oh apa ini mengepul ya
soalnya kan „steam‟, „steam‟ biasanya mengepul. Terus itu ternyata gambar
selanjutnya memperjelas, itu si Shiro lagi megang cup ramen. Dan itu ada uap-uap
yang lagi mengepul di situ.”
(“I guessed from the sentence, „Steam wafts lazily from the cup, and I blow gently
on it.‟ From that sentence I think that „wafts‟ means ‘mengepul’, because there‟s a
word „steam‟. After that there‟s a picture that made the portrayal clear; Shiro,
holding a cup of ramen, and the steam was wafting from it.”) From this statement,
it could be concluded that Della used visual mode in learning the word „wafts‟,
which was the steam from Shiro‟s cup ramen when he was holding it.
From the examples above, it can be seen that visual element could be used
in vocabulary learning through visual novel. The result corresponds with
Pressley‟s (1977) theory, that if a text is presented with related pictures, it will
leave more profound print in learner‟s mind. It is also in line with Underwood
(1989) who suggests that pictures are remembered better than texts, and texts are
remembered better if they are strongly related with the pictures which are
11
presented. Therefore, the role of visual in visual novel is not only to present words
in contextual picture, but also to make the new vocabulary easily understood by
the audience.
It is possible that two or three focuses are used in learning the same word,
depending on what pictures which appear in the contexts. Sometimes, another
element was also used to aid the participants in learning, however, two elements
coming up together could show that multimodal learning can be found in visual
novel, which will be discussed later.
Textual
Textual element takes form in text. It is the most important attribute in
visual novel because audience grasps the whole information of the story through
it. For example, Shin used the aid of written element in 15 unknown words, Yesha
in 23 words, and Della in 10 words (see Appendix A). Textual mode could be
categorized into two focuses; focus on form and focus on story.
Textual mode with the focus on form. This focus was used by the
participants through identifying clues from words, clause, and/or sentences that
surrounded the unfamiliar words. For example, all participants learned the word
„throng‟ as ‘kumpulan’ or ‘kerumunan’. The participants that used the focus on
form in learning the word „throng‟ were Shin and Della, while Yesha used both
focuses. Shin said, “Ini ada „throng‟. „Throng of students‟. Guessingnya sih kirakira-nya kayak „kerumunan‟ gitu, terus tadi buka kamus, iya bener artinya
„kerumunan‟.”
(“„Throng‟. „Throng of students‟. I think it‟s more like
12
‘kerumunan’. I also checked on the dictionary, and it‟s correct.”) The other
participant, Della, said, “Di sini ada „merging with the throng of students‟. Jadi
„kumpulan‟” (“There‟s a sentence „merging with the throng of students‟, so I
think it means ‘kumpulan’.”)
Both of them said that they learned the word „throng‟ through the sentence
„merging with the throng of students‟. From these examples, they showed how
words surrounding the unknown word could help them in learning the meaning
even though there was no visual or aural aid accompanying the textual mode.
Textual mode with the focus on story. This focus was used when the
participants recalled the story illustrated by the text to aid them in understanding
unknown vocabulary. The example was when Shin learned the meaning of
„brash‟, she said:
“Dari ceritanya, kan Alistair ni ngajak gelut kan, masak dia nyampah terus…
ngerebut item, padahal kan harusnya itu punyaku, terus kan juga aku yang kerja
keras. Terus kan aku bilang, aku mau „revenge‟. Terus si Fiona-nya ini takut kalau
aku mau ngelakuin „something stupid‟ gitu.”
(“I recalled the story, so Alistair was trying to pick a fight with me, he killed the
enemy that I was fighting to... and he ended up getting my reward item. After that,
my character said, „I want revenge‟. Then Fiona‟s afraid that I might do
„something stupid‟.”) Here, unlike in the previous focus that the clue is gotten
from the form of sentence, Shin recalled a part of the story that gave her clues to
learn the meaning of „brash‟.
13
Textual mode with the focus on form and story. It is also possible that
the two focuses are used in learning the same word. The example was when Della
learned the meaning of „PvP‟. During the interview, she said:
“„PvP‟. Guessing dari gambar, dari konteks. Soalnya kan awalnya si Alistair
ngerebut batunya Rui, terus mau duel, nah PvP itu aku nebak „Player versus
Player‟. Terus ada kata „match‟ juga, jadi itu kan mesti „gelut‟.”
(“„PvP‟. I guessed from the picture and context. At the beginning, Alistair stole
Rui‟s stone and they wanted a duel, so I guessed PvP means „Player versus
Player‟. There‟s a word „match‟, too, so it must be something related to „battle‟.”
In conclusion, Della used both focuses in learning the word „PvP‟; the story
illustration (that Alistair stole Rui‟s stone leading to a duel) and the sentence “The
PvP match you participated in with player Alistair has been cancelled”.
These focuses were in line with Kintsch (1988) suggestion, that “learning
with text consists of the construction of two interconnected mental representations
(i.e., a textbase and a situation model)” (cited in Zarei & Gilanian, 2013). The fact
that written mode was used by the three participants in most of the learning
process could prove the importance of text mode in visual novel, thus may
accompany the other elements or stand alone to be used as an aid in understanding
unfamiliar words.
Aural
Aural mode, the last multimodal element, is anything in multimedia which
can be heard by the ears as the audio sensory perception. In visual novels, usually
it can be found in characters‟ voices, background music, and sound effects. In this
visual novel, aural mode could be found through background music and sound
14
effects. Thus, these two became the focuses from aural mode. The only participant
that used aural mode was Yesha. She used it in 2 words (see Appendix A).
Aural mode with the focus on background music. This focus was used
when the participants focused on the music while playing and used it in learning
the meaning of unknown words. Background music may change depending on the
situations, moods, and some places and characters have their own music theme.
Nonetheless, Yesha did not use this focus at all. Instead, she used the other focus;
focus on sound effects.
Aural mode with the focus on sound effect. This focus was used when
the participants focused on the sound effects that appeared during the emergence
of unknown words. The example was when Yesha learned the meaning of „dash
off‟. Yesha said:
“„Dash off‟ itu aku guessing-nya lari pergi. Ada kata-kata ini, „I grab my books
and dash off to class.‟ Dari konteks kalimatnya aja, jadi kayak dia ambil buku
terus pergi. Terus kan ada suara bell masuk nya juga kan itu, jadi dia pergi buat ke
kelas.”
(“I think „dash off‟ is similar to ‘lari pergi’. There‟s a sentence „I grab my books
and dash off to class.‟ From the context I got the information that she took her
books then went off. Then there‟s the sound of the bell ringing, so she went to
attend the next class.”)
Based on the previous studies, it is believed that multimedia presentations
including background music improve more learning than in silent condition
(Groot, 2006; Thiessen & Saffran, 2009, cited in Phan & Cheon, 2013). However,
from the interviews with the three participants, it could be concluded that aural
15
mode in visual novel was not as helpful for the participants in understanding
unknown vocabulary. Yesha explained:
“Kalau suara, menurutku kok paling susah buat diandalin, karena kan kalau di
game ini cuma ada music latar sama efek suara, itu kan ya nggak pasti keluar pas
vocab yang nggak ngerti itu muncul kan. Terus kita interpretasinya juga tidak akan
sejelas gambar sama teks, kayak kalau misalkan ceritanya sedih musiknya juga
sedih, tapi mungkin buat aku sih musiknya bisa aja nggak sedih-sedih amat, jadi
kan belum tentu exact music-nya itu bisa menggambarkan tentang situasi yang
sebenernya.”
(“I think aural mode cannot be used as much, because in this game there are
background music and sound effects only, and they do not always appear when
the unknown vocabulary appears. We also cannot interpret audio element as clear
as visual and written elements, as example when the story‟s mood is sad then
maybe the music is also played in sad tone, but maybe for me the music is just
plain, so its ability to portray the exact situation is quite uncertain.”) From this
statement it showed that aural mode offers limited aids which resulted in
participant‟s limited understanding.
Shin also said, “Mungkin kalo misalnya ada suara orangnya ngomong
mungkin bisa lebih kebantu dari tone-nya dia.” (“Maybe, if there‟s any character‟s
voice, it can aid us through the tone.”) In this case, there are other visual novels
which offer more aural modes including character‟s voices, which can be used in
learning vocabulary. However, when asked whether she would like to play a
visual novel that does not provide any aural mode, Della said, “Sebenernya nggak
asik sih, soalnya suara juga mendukung mood, tapi untuk vocabulary nggak
terlalu crucial.” (“Actually I think that it‟s not fun, because the audio also boosts
the mood, but in vocabulary learning it‟s not very crucial.”) It shows that even
though she thought that audio element could not be used significantly in
16
vocabulary learning, she thought that audio element enhanced the mood and made
the visual novel more enjoyable.
Multimodal Elements in One Word Learning
Multimodal learning (Massaro, 2012) happens when two or three
representations appear in such context at the time the unknown word appears that
help the audience in learning the meaning. How each mode was used by the
participants in their vocabulary learning has been discussed, and this section
discusses how the three modes; visual, textual and aural were used in learning the
same word.
An example of how participant used the three modes in learning a word
was from Yesha, when she learned the meaning of „echoing boom‟. She said:
“„Echoing boom‟ itu kayak suara dribble-an dari basket, boom boom boom… Kan
tadi juga ada scene pas Derek main basket, suasananya keliatan dari gambarnya
tuh lagi di gym, terus ada suara boom boom boom dari main bola basketnya
juga… Kan ceweknya ini kan ke gym, terus kalimatnya „I can hear shouts and
echoing boom‟, jadi guess dari situ.”
(“‟Echoing boom‟ is like the sound of basket ball being dribbled, boom boom
boom... There was also a scene where Derek played basketball, the ambience was
shown from the picture that the characters were in a gym, then there was a sound
effect „boom boom boom‟ from the ball, too… The main character was going to
the gym, and there was a sentence „I can hear shouts and echoing boom‟, so I
guessed from those clues.”) Here, she mentioned „picture‟ as an instrument in
visual mode, „sound effect‟ as in aural mode, and „sentence‟ as in textual mode. It
means that Yesha used all of the multimedia elements in learning the unknown
vocabulary.
17
Error in Multimodal Learning Environment
Error here means when the audience misunderstand the word meaning due
to either limited clue that multimodes offer or audience‟s lack of understanding.
This error only occured in Yesha, with 7 errors (see Appendix A). For example,
Yesha learned the meaning of „spotty internet‟ as „a spot for accessing internet‟.
During the interview, she said:
“Spotty itu aku ingetnya kata „spot‟ dan spot itu kayak titik, satu tempat buat
kamu... Kayak istilahnya apa ya, koneksi wi-fi gitu lho, kalo aku guessing-nya sih
itu. Soalnya di sini kan ada gambar suasananya tuh lagi di ruangan computer gitu,
banyak komputernya. Nah spotty internet itu nggambarin tempatnya ini, spot buat
internetan.”
(“The word „spotty‟ made me think that it‟s like a „spot‟, so it is a place where
you… What‟s the term? It‟s like „wi-fi connection‟, in my opinion. It‟s because
there‟s a picture that portrayed the ambience, it‟s like a computer lab since there
were many computers. So the word „spotty internet‟ refers to the place, a place to
access the internet.”) From this statement, it was concluded that Yesha said that
because the word „spotty internet‟ appeared when the main character was shown
being in a computer lab. However, the meaning of „spotty internet‟ (from the
sentence “still, darn this school‟s spotty internet connection”) was more like the
performance of the internet connection which was „not always good‟. It could be
seen that multimodes could not always promising for true and direct meaning as
in dictionary, because multimodes here only facilitating clues to be used by the
participants. Therefore, the meaning of the unknown words resulted from the
participants‟ own thinking process.
18
Vocabulary Learning in Visual Novel
Visual novel can be considered as one of multimedia that is rich in
vocabulary due to its written element, and has visual and aural elements to help
portray the story. The advantage of learning vocabulary through visual novels
besides it enhances incidental learning and has multimodal elements is that its onclick setting. It means that the next text, followed by related pictures and sounds
appear only after the audience clicked the application. Therefore, the audience can
focus on the current event and grasp the meaning of the sentences, thus improve
vocabulary learning. This advantage does not correspond in other media such as
novels which have lots of text that may distract the reader‟s focus, and videos that
keep on going except they are being paused by the audience.
However, there is also ineffectiveness in visual novels regarding their role
as multimedia. Visual representations in visual novels are quite limited because
the pictures only change from one picture to another – unlike videos that provide
motions. Moreover, unlike visual element, aural element does not have crucial
role in visual novel. Therefore, some other visual novels might not provide any
audio at all. Just like its name; it is presented visually (as in visual) and textually
(as in novel), and aural is merely optional – it is still called „visual novel‟ even
when it does not have any aural element in it. The participants also said that aural
mode (in the form of background music and sound effects only) was the least
useful mode in learning vocabulary. There were two possibilities; the first was
because the sound effects and background music did not always appear where the
unknown word took place, and the second was because aural mode is more
19
intangible than the other modes. Nevertheless, music and sound effects in
multimedia might make the learning process become more enjoyable to learners,
thus incidental learning may happen (Weiner, 1990).
CONCLUSION
This research was conducted to see the types of multimodal elements in
visual novel that can be used by learners in learning English vocabulary. Based on
the data gathered through SRP method, it could be seen that every time unknown
word appeared, participants could make use of multimodal elements in the visual
novel, which resulted in incidental learning. Textual mode in visual novel had the
same nature as in general novel, and could be used to learn vocabulary meaning
through sentence form and story plot. Aural mode in this visual novel could only
be found in background music and sound effects, which were quite intangible to
be used in understanding unknown words, and did not always appear when the
unknown words appeared. Meanwhile visual mode could help in portraying the
scene (time and places), characters‟ expressions, and events, and used by the
participants in seeing the contexts – an advantage of visual novel compared to
common novel which only provides text. Although multimodes provide clues for
the participants to be used in learning unknown words, sometimes the meaning
that the participants got were not in direct meaning, but rather in close meaning
(not the exact synonym word, but they understand the meaning) or even resulted
in error.
20
The significance of the study was to give an insight that video games
could give meaningful purpose besides for fun; to acquire English vocabularies
through incidental learning. It was done due to the claim that video game could
increase gamers‟ English skill, although many people often see video games as a
pleasure provider only, not as a tool to provide learning environment. The
findings of the study might motivate people to use visual novel as a medium to
learn English vocabulary.
However, this research was still far from perfect due to some limitations,
which could be put into considerations for further research. There were only 3
participants that were involved in this research, so the result that visual novel can
be used as a medium in learning vocabulary might not be generalizable in
different context. Another problem was in getting the suitable visual novel to be
used. The visual novel that was used in this research, “RE:Alistair++” only
provided background music and sound effect as the aural mode, which could not
be used by the participants well in understanding the unknown vocabulary. It
would be better if the visual novel also provides characters‟ voices so it can be
used by learners more in understanding the tone and pronunciation. The
vocabulary level also has to be adjusted with the participants. If it is too easy, then
there may be no vocabulary learning at all. The content of the visual novel also
has to be considered. Sometimes, visual novels contain mature contents, such as
sexual and violence. Considering all of that, getting a visual novel that suits well
for the research is not easy. Therefore, using other types of video games or even
21
other multimedia can be an option to conduct a further research on vocabulary
learning in multimodal environment.
ACKNOWLEDGEMENT
First, I would like to thank God for giving me strength throughout
completing this thesis. I would also like to express my gratitude to my research
supervisor, Mr. Toar who always guided and gave me solutions, and my research
examiner, Prof. Astika who was willing to give me suggestions and corrections.
Without their assistance, this paper would not be as successful. My gratitude also
goes to my family and friends for their encouragement every time I feel down. I
thank my brother too, for lending me his laptop during my thesis writing because
mine was broken. For Hasenvan, thank you for suggesting me to use the visual
novel. I did not know this video game would become the topic of my thesis. Last
but important, I want to thank my participants who gave me their time and efforts
during data collecting process. Without their contribution, this paper would have
never been done.
REFERENCES
Ainsworth, S., & Van Labeke, N. (2002). Using a Multi-representational Design
Framework to Develop and Evaluate a Dynamic Simulation Environment.
Retrieved
May
1,
2015,
from
http://www.nvl.calques3d.org/publications/2002.DynVis.Framework.pdf
Deci, E. L., Vallerand, R. J., Pelletier, L. J., & Ryan, R. M. (1991). Motivation
and Education: The Self Determination Perspective. Educational
Psychologist, 26 (3 & 4), 325-346.
22
deHaan, J., Reed, W. M., & Kuwada, K. (2010). The Effect of Interactivity with
a Music Video Game on Second Language Vocabulary Recall. Language
Learning & Technology, 14, 1094-3501.
Gass, S. M., & Mackey, A. (2000). Stimulated Recall Methodology in Second
Language Research. Retrieved May 1, 2015, from
http://www.cc.kyoto-su.ac.jp/information/tesl-ej/ej16/r18.html
Gu, P. Y. (2003). Vocabulary Learning in a Second Language: Person, Task,
Context and Strategies. Retrieved October 5, 2014, from Teaching English
as a Second or Foreign Language Web site:
http://www.tesl-ej.org/ej26/a4.html
Huyen, N. T. T., & Nga, K. T. T. (2003). Learning Vocabulary through Games:
The Effectiveness of Learning Vocabulary Through Games. Retrieved
September 21, 2014, from Asian EFL Journal Web site:
http://www.asian-efl-journal.com/dec_03_vn.pdf
Kim, D., & Gilman, D. A. (2008). Effects of Text, Audio, and Graphic Aids in
Multimedia Instruction for Vocabulary Learning. Educational Technology
& Society, 11 (3), 114-126.
Kozma, R. B. (1991). Learning with Media. Review of Educational Research,
61(2), 179-212.
Leu, D. J. (2002). Internet Workshop: Making Time for Literacy. The Reading
Teacher, 55(5), 466-72.
Massaro, D. W. (2012). Multimodal Learning. Retrieved November 14, 2014,
from https://wss.apan.org/jko/mls/Learning%20Content/Massaro%20%20Multimodal%20Learning%20(Encyclopedia%20of%20the%20Scienc
es%20of%20Learning).PDF
Mayer, R. E. (2003). Elements of a Science of E-learning. Journal of Educational
Computing Research, 29(3), 297-313.
Meihami, H., Meihami, B., & Varmaghani, Z. (2013). CALL in the Form of
Simulation Games: Teaching English Vocabulary and Pronunciation
through Sims. International Letters of Social and Humanistic Sciences, 8,
2300-2697.
Moreno, R. (2002). Who Learns Best with Multiple Representations? Cognitive
Theory Implications for Individual Differences in Multimedia Learning.
Retrieved
May
1,
2015,
from
http://www.editlib.org/p/10238/proceeding_10238.pdf
Nation, P. (2002). Best Practice in Vocabulary Teaching and Learning. J. C.
23
Richards & W. A. Renandya (Eds.), Methodology in Language Teaching:
an Anthology of Current Practice (pp.267-272). London: Cambridge.
Nation, P. (2005). Teaching Vocabulary. Retrieved November 24, 2014, from
Asian EFL Journal Web site:
http://www.asian-efl-journal.com/sept_05_pn.pdf
Phan, V., & Cheon, J. (2013). Multimedia Learning: Effects of Background Music
and Sound Effects on Space Science Concept Learning. Retrieved March
29, 2015, from www.aect.org/pdf/proceedings13/2013/13_23.pdf
Pressley, M. (1977). Imagery and Children's Learning: Putting the Pictures in
Developmental Perspective. Review of Educational Research, 47(4), 585622.
Sankey, M., Birch, M., & Gardiner, M. (2010). Engaging Students through
Multimodal Learning Environments: the Journey Continues. Retrieved
November 14, 2014, from
http://ascilite.org.au/conferences/sydney10/procs/Sankey-full.pdf
Thornbury, S. (2002). How to Teach Vocabulary.
England: Pearson Education Limited.
Turgut, Y., & İrgin, P. (2009). Students’ Experiences of Computer Games and
ELL: from Phenomenological Perspective. Retrieved September 9, 2014,
from http://dbe.metu.edu.tr/convention/proceedingsweb/Games.pdf
Underwood, J. (1989). Hyper Card and Interactive Video. CALICO Journal, 6 (3),
7-20.
Weiner, B. (1990). History of Motivational Research in Education. Journal of
Educational Psychology, 82 , 616-622.
Zarei, A. A., & Gilanian, M. (2013). The Effect of Multimedia Modes on L2
Vocabulary Learning. International Journal of Management and
Humanity Sciences., 2 (S), 1011-1020.
24
APPENDIX A
Table of Participants’ Unknown Words, Multimodals Used and Actions
*: error in processing meaning
Participants
Shin
Yesha
Words
Multimodals
Used
AP & MP
forfeited
huddle
textual, visual
Textual
Textual
recluse
menial
brash
throng
shallow
postulated
bracket
noutorious
pristine
prudent
text, visual
Textual
textual
textual
textual
textual
textual
textual
textual
textual
seethe
imposing
textual, visual
textual
Actions
focus on form (textual), focus
on event (visual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual), focus on character's
expression (visual)
focus on form (textual)
focus on story (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual)
blessed stone
spotty
internet*
visual
focus on event (visual)
visual
forfeited
textual, visual
dash off
scribble
menial
textual, aural
textual
textual
smirking
brash
rotten*
textual, visual
textual
textual
focus on place (visual)
focus on story & form
(textual), focus on place
(visual)
focus on form (textual), focus
on sound effects (aural)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual)
focus on form (textual)
25
Della
echoing
boom
something of
a stretch*
merging
throng
heads off
textual, visual,
aural
drag*
postulated
bracket
textual
textual
textual
sighs*
textual, visual
chilling
textual, visual
chuckles
wipes off
glance
sensible*
picky*
visual
textual
textual
textual
textual
waft
page
textual, visual
textual
textual
textual
textual
textual
menial
textual
PvP
textual, visual
brash
throng
postulated
social butterfly
textual
textual
textual
textual
sketchy
spacebook
savvy
textual
textual
textual
waft
textual, visual
26
focus on form (textual), focus
on place (visual), focus on
sound effects (aural)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual), focus
on place (visual)
focus on event & character's
expression (visual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual), focus
on event (visual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual), focus on event
(visual)
focus on story & form
(textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form and story
(textual)
focus on form (textual)
focus on form (textual)
focus on form (textual), focus
on event (visual)
APPENDIX B
Transcription of the Interviews with Participants
Transcription 1
W: Wenda
S: Shin
W: Selamat sore.
S: Sore.
W: Oke tadi kan sudah main visual novel, Alistair. Ada kata atau vocab yang
belum pernah tau atau baru dipelajari di game ini?
S: Iya, ada beberapa.
W: Kebanyakan guessing atau pakai kamus?
S: Kalau kata-kata yang bisa nebak artinya kira-kira apa gitu aku nebak dulu, terus
buka kamus buat mastiin artinya. Soalnya kalau nggak gitu nanti takutnya salah
milih options.
W: Oke. Tadi katanya nggak tau yang „AP‟ sama „MP‟. Tapi ada gambaran nggak
„AP‟ sama „MP‟ itu apa?
S: Karena tadi tulisan „AP‟ sama „MP‟-nya pakai huruf besar semua, jadi…
mungkin itu… ini sebenernya guessing ya, tapi kalo menurutku ini…. Vocab
sekitar game-game gitu. Soalnya kan ini juga lagi main game, main game online
gitu. Kalo misalnya attack, atau defense gitu kan pakai „ATK‟, „DEF‟ kan huruf
besar semua, kalo ini kan juga tadi „AP‟ sama „MP‟ pake huruf besar semua, jadi
ya itu mungkin ya sodara-sodaranya „ATK‟, „DEF‟ gitu.
W: Jadi guessing-nya itu clue-nya dari bentuk kalimat?
S: Iya, bentuk kalimat sama bentu katanya sendiri itu.
W: Taunya itu lagi main game dari mana?
S: Itu screen-nya lagi main game.
W: Oke. Nemu kata apa lagi yang susah?
S: Ini bacanya apa ya, „forfeited‟.
W: Untuk tau itu artinya, gimana?
27
S: Ini sih otomatis guessing, mungkin artinya kayak „mundur‟ gitu, mundur dari
permainan, soalnya kan kalau ngeliat dari kalimat sebelumnya internetnya nge-lag
gitu, terus kalimat sesudahnya ini juga kayak ngomongin efek dari kalimat yang
tadi sebelumnya itu. Jadi aku nyimpulin kalo gara-gara internet-nya nge-lag,
makanya terpaksa „mundur‟, karena itu juga kan pa
VOCABULARY LEARNING THROUGH VISUAL NOVEL
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Wenda Insani Wangi
112011003
ENGLISH LANGUAGE EDUCATION PROGRAM
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2015
MULTIMODALITY IN SIMULATION GAMES:
VOCABULARY LEARNING THROUGH VISUAL NOVEL
THESIS
Submitted in Partial Fulfillment
of the Requirements for the Degree of
Sarjana Pendidikan
Wenda Insani Wangi
112011003
ENGLISH LANGUAGE EDUCATION PROGRAM
FACULTY OF LANGUAGE AND LITERATURE
SATYA WACANA CHRISTIAN UNIVERSITY
SALATIGA
2015
i
PUBLICATION AGREEMENT DECLARATION
As a member of the (SWCU) Satya Wacana Christian University academic
community, I verify that:
Name:
Student ID Number:
Study Program:
Faculty:
Kind of Work:
Wenda Insani Wangi
112011003
English Language Education Program
Faculty of Language and Literature
Undergraduate Thesis
In developing my knowledge, I agree to provide SWCU with a non-exclusive
royalty free right for my intellectual property and the contents therein entitled:
Multimodality in Simulation Games: Vocabulary Learning through Visual Novel
along with any pertinent equipment.
With this non-exclusive royalty free right, SWCU maintains the right to copy,
reproduce, print, publish, post, display, incorporate, store in or scan into a
retrieval system or database, transmit, broadcast, barter or sell my intellectual
property, in whole or in part without my express written permission, as long as my
name is still included as the writer.
This declaration is made according to the best of my knowledge.
Made in
Date
: Salatiga
:
Verified by signee,
Wenda Insani Wangi
Approved by
ii
iii
iv
TABLE OF CONTENT
Inner cover ............................................................................................................... i
Approval page ......................................................................................................... ii
Publication agreement declaration ......................................................................... iii
Copyright Statement …………………………………………………………..…iv
ABSTRACT ............................................................................................................ 1
INTRODUCTION................................................................................................... 1
Vocabulary learning ............................................................................................ 2
Advantages of video games in vocabulary acquisition ....................................... 4
Multimodes in visual novel and their role in vocabulary learning ...................... 5
THE STUDY ........................................................................................................... 6
Research Question ............................................................................................... 6
Context of Study .................................................................................................. 6
Participants .......................................................................................................... 7
Instrument of Data Collection ............................................................................. 7
Data Collection Procedure ................................................................................... 7
Data Analysis....................................................................................................... 8
FINDINGS AND DISCUSSION ............................................................................ 8
Visual................................................................................................................... 9
Visual mode with the focus on place. ............................................................ 10
Visual mode with the focus on character‟s expression.. ................................ 10
Visual aid with the focus on event ................................................................. 11
Textual ............................................................................................................... 12
Textual mode with the focus on form. ........................................................... 12
Textual mode with the focus on story. ........................................................... 13
Textual mode with the focus on form and story ............................................ 14
Aural .................................................................................................................. 14
Aural mode with the focus on background music ......................................... 15
Aural mode with the focus on sound effect. .................................................. 15
Multimodal Elements in One Word Learning ................................................... 17
Vocabulary Learning in Visual Novel ............................................................... 19
v
CONCLUSION ..................................................................................................... 20
ACKNOWLEDGEMENT .................................................................................... 22
REFERENCES ...................................................................................................... 22
APPENDIX A ....................................................................................................... 25
APPENDIX B ....................................................................................................... 27
LIST OF TABLE
Table 1: Frequency of Multimodal Modes Used.....................................................9
vi
Multimodality in Simulation Games:
Vocabulary Learning through Visual Novel
Wenda Insani Wangi
ABSTRACT
This research was an analysis to find out the types of multimodes in
simulation game, especially visual novel that can be used in English vocabulary
learning. This research was done due to several claims that learning vocabulary
through video game is effective. The theory that was used to analyze this research
was the theory of Multimodal Learning Environment (Sankey, Birch & Gardiner,
2010), which promote multimodes (visual, aural, and textual) in multimedia as a
tool in learning. The researcher used descriptive approach to find out participants‟
actions in making use of the multimodes in dealing with unknown vocabulary
while playing visual novel, with using Stimulated Recall Protocol method and
interview as the instrument. There were 3 students of batch 2011 from Satya
Wacana Christian University‟s Faculty of Language and Literature as the
participants. It was found that the participants could make use of multimodes in
visual novel in vocabulary learning, although not all of the learning process
resulted in correct meaning. This research may give an insight for the readers to
consider using visual novel as one of multimedia to learn vocabulary and promote
video games as a tool in multimodal learning environment.
Keywords: vocabulary learning, incidental learning, multimodes, visual novel,
simulation game
INTRODUCTION
The claim that video games increase gamers‟ English proficiency has been
spread for a long time, especially in non-native speakers. It is true that people do
not play video games to study, or even do not have any intention to learn while
playing games. Still, the claim stays and constantly declared by gamers, which
provokes pros and cons from society.
1
Regarding to the claim, Meihami, Meihami, & Varmaghani (2013) suggest
that Video games prove potential in promoting language learning and not just for
engaging and entertaining the users. The successfulness of learning vocabulary
through video games might be affected by the existence of multimodes, which is
supported by the theory of multimodal learning (Massaro, 2012). Presentation of
new vocabulary in contexts and multiple exposures to provide clues to meaning
are important for learners to learn new vocabulary. Thus, multimodality in
audiovisual aids which present contextual sounds and illustrative pictures provide
great opportunity for vocabulary learning.
This study was conducted to understand in what way multimodes can be
used by learners in their vocabulary acquisition. The focus of this study was to see
the types of multimodes that learners used in finishing the visual novel every time
they had to deal with unfamiliar words. This paper presents the findings of the
research to show how the participants could make use of multimodes in
vocabulary learning. Moreover, this study would provide more evidence on how
multimodal material could help students in learning vocabulary.
Vocabulary learning
English as a Foreign Language learners usually use various strategies to
memorize vocabulary words, such as word lists (Kim & Gilman, 2008). One of
the ordinary methods in memorizing is learning vocabulary deliberately, which is
also one of the least efficient ways of learning vocabulary (Nation, 2005).
Learning vocabulary through memorizing is quite difficult for students because
they cannot see the real context of each word. DeCarrico (2001) also states that
2
words should not be learnt separately, nor by memorizing without understanding
(cited in Huyen & Nga, 2003).
Thornbury (2002) also suggests that second language learner usually use
their mother tongue as a standard to the target concept, where the new vocabulary
is simply mapped directly onto L1 equivalent. Thus, acquiring vocabulary will be
much more effective if the context which the word takes place can be seen. When
the context can be seen, it is possible to guess the word‟s meaning with the help of
mother tongue, and later learners can use the word for making their own new
sentences. If learners only learn vocabulary by memorizing and do not know how
to use them contextually, then learners only know how to use the vocabulary in an
exact form, but they do not know how to use it in different context in real life
communication. It is because when an exact word is separated from a sentence
and being put in another sentence which has different context, then the word may
change its meaning (related with the study of semantic). That is why both
remembering words and the ability to use them automatically in a wide range of
language contexts become the purpose of vocabulary learning (McCarthy, 1984,
cited in Gu, 2003).
To avoid deliberate vocabulary learning, Krashen (1989) suggests to learn
vocabulary incidentally which is believed to achieve better results than intended
one, which is learned through vocabulary listing - which is out of contexts and not
supported by any aid to help learners guess or see the meaning (cited in Gu,
2003). Nation (2002) also suggests that incidental learning through listening and
reading using context clues can help learners to figure out the meaning of the
3
word. Incidental learning happens when learners „do not realize‟ that they are
currently learning. Here, video games can take position as a medium for incidental
learning.
Advantages of video games in vocabulary acquisition
Nowadays, students are not really fond of reading books, especially when
the book is presented in L2. According to Leu (2002), students who are not fond
of reading books and writing with pencil and paper prefer to learn with the use of
technology. Video games can be one of the technologically-provided media that
can support language acquisition process, which is proved through the study on
the effect of simulation games on sailor and mariners‟ English vocabulary and
pronunciation learning of Khoramshahr Navy University, Iran (deHaan, Reed, and
Kuwada, 2010). The results showed that simulation games had a significant
impact on learners, in which learners who learned English through Navy
Simulator Game English Learning Program performed much better than learners
who only learned English through ordinary method.
The motivational role in vocabulary acquisition process through
simulation game is supported by “self-determination theory: the role of pleasure,
joy, and curiosity in motivating people to do something” (Deci, Vallerand,
Pelletier, and Ryan, 1991). Playing video games usually is done in order to have
fun, so the “burden” to learn is blurred because learners do not realize that they
are currently learning L2 through reading while playing.
The role of video game in vocabulary learning is also supported by Gass &
Selinker (2001) that learning language through simulation video games is
4
contextualized, therefore the learners can acquire language in a natural setting
with various situations which prepare the learners to use the acquired language
properly (cited in Turgut & Irgin, 2009). These contextual aspects are supported
by multimodes in video games.
Multimodes in visual novel and their role in vocabulary learning
Some media, like television, provide instructional elements which are
presented in more than one sensory mode, which may be in the form of visual,
aural, and written – hence, “multimodes” (Sankey, Birch & Gardiner, 2010). It is
believed that the use of multiple representations has been recognized as a very
strong method to smooth the progress of understanding (Moreno, 2002). Kozma
(1991) suggests that multimedia can facilitate learning because the different
advantages of each mode are being put together in one instructional environment.
There is a term called „multimodal learning‟ which supports the theory, where in
the learning situation there are multiple sensory systems which influence learner‟s
actions to understand the unknown words (Massaro, 2012). Mayer (2003) also
argues that combination of words and pictures develops more profound learning,
known as the „multimedia effect‟. In case that an audience cannot understand the
text as the input, the visual representation might aid them to understand the
meaning (Ainsworth & Van Labeke, 2002).
Video games can be included as one of multimedia, and it may facilitate
learning. Although there are lots of game genres, not all video games provide
story plot which is rich of vocabulary for learners. However, genres that provide
story plot and different endings based on the options the audience made during the
5
playtime, engage more in the audience‟s role in incidental learning. That is why
this study used visual novel (which is a sub-genre for simulation game); it is like
reading a novel – rich of vocabularies, but with gaming background. By using
visual novels, learners can focus more in processing vocabulary meaning because
the contexts have been provided by the aids of sounds, texts, and pictures. These
multimodes present additional representations to help learners in understanding
unknown vocabularies. Therefore, visual novels are believed to contribute in
learners‟ L2 acquisition – they provide visualization of the event (pictures), texts
(narratives, dialogues) to observe, and aural (background music, sound effects) to
enhance the vocabulary acquisition.
THE STUDY
Research Question
The study was conducted to answer the following question: “What are the
multimodes in Visual Novel that can be used by learners in understanding
unknown vocabulary?”
Context of Study
This study was a descriptive research which was conducted in January
2015. It was conducted in informal situation. The study had the objective to see
the types of multimodes in visual novel that can be used by learners in learning
English vocabulary. The research used visual novel named “RE: Alistair++” by
Sakevisual, a dating simulation game. The time estimated to finish one route was
around 1 to 2 hours depending on the participants‟ speed in reading and the route
6
itself. With many options and endings the game provided, the chance that
participants encountered different routes was high.
Participants
The researcher chose 3 students of batch 2011 from Satya Wacana
Christian University‟s Faculty of Language and Literature as the participants. The
consideration to take students from batch 2011 was because the last year students
might understand more about reading in English, so the research could focus only
on unfamiliar words. The researcher would use pseudonymes for the participants‟
identity protection.
Instrument of Data Collection
In collecting the data, the researcher used Stimulated Recall Protocol
(SRP) method and interview as the instrument. SRP method is used to investigate
participants‟ thoughts and strategies during doing a task given through asking
participants to report them immediately after they have completed the task, by
presenting tangible visual and aural reminder in the form of video recording of
their performance on the task to help them remember their thought processes
(Gass & Mackey, 2000).
Data Collection Procedure
First, the researcher chose participants from FLL batch 2011 to play the
game from the beginning until the end (approximately 1-2 hours) one by one in
different times. After that, researcher recorded the screen and the audio, and
observed the participant during the playtime. Then, after the participant finished
7
the game, the researcher used SRP method by playing the video recording and
observed it again with the participant. After that, the researcher interviewed the
participant about the multimodes that they used to learn the unfamiliar words
during playing.
Data Analysis
First, the researcher transcribed the interviews into clean transcriptions.
Then, the researcher categorized the modes that were used by the participants in
understanding the unknown vocabulary. The last, the researcher analyzed the data
through the study of Multimodal Learning Environment (Sankey et al, 2010).
FINDINGS AND DISCUSSION
Through the SRP sessions with the three participants – Shin, Yesha, and
Della, the researcher managed to find out how they were able to use the
multimodes in learning English vocabulary from visual novel in details. Based on
Sankey et al‟s (2010) perspective on Multimodal Learning Environments, it was
observed that in “RE: Alistair++” there were 3 instructional elements in different
sensory modes; visual, aural, and written, that were used by the participants in
order to understand the meaning of the unknown words.
8
Table 1: Frequency of multimodes used
Modes
Frequency
%
Visual
14
22
Aural
2
3
Textual
48
75
Total
64
100
As shown in Table 1, the frequency of multimodes used in total was 64 times. The
total number of the unknown words was 51 (see Appendix A). In understanding
some words, each participant used more than one mode, which resulted in the
difference between the total number of the modes used and the total number of the
unknown words. Below is the discussion on how the modes were used by
participants in dealing with the unknown words.
Visual
Visual mode is anything in multimedia which can be seen by the eyes, as
visual sensory perception. In visual novels, usually it can be found in characters‟
physical appearances, characters‟ expressions, situation of place and time, and so
on. Visual mode is the important element that differs „visual novels‟ from the
usual novels, because in visual novel there are visual representations in context
that might help us portraying the scene. Without visual mode, it is just a novel.
9
While playing the game, Shin used the aid of visual mode in 3 unknown
words, Yesha in 9 words, and Della in 2 words (see Appendix A). Furthermore,
the researcher would categorize the visual aid based on three focuses; place,
character‟s expression, and event.
Visual mode with the focus on place. This focus was used when the
details of place was portrayed through image (usually the background picture) and
used by the participants to understand the unknown words. The example was from
Yesha. She said,
“„Chilling‟ di kalimat „just chilling after a long day‟. Si ceweknya itu tanya
„gimana kabarmu‟ gitu, terus cowoknya bilang „not bad‟, „chilling‟ itu mungkin
„refreshing‟ makanya dia jalan-jalan ke mall. Terus ini gambar tempatnya juga di
mall.”
(“‟Chilling in „just chilling after a long day‟; the girl asked „how are you‟ and the
boy said „not bad‟, „chilling‟ here maybe means „refreshing‟ so he went to the
mall. There‟s a picture of the mall as the background.”) From this statement, it
was concluded that Yesha said that because the word „chilling‟ appeared when the
characters were shown being in the mall.
Visual mode with the focus on character’s expression. This focus was
used when the characters‟ expressions were used by the participants to understand
the unknown words. Characters‟ expressions may change depend on the story and
mood, so each character has various expressions. The example was when Shin
learned the meaning of „seethe‟. She said, “Kalo guessing tuh menurutku artinya
„marah‟. Liat dari ekspresinya, sama kalimat sebelumnya kan juga aku
nyinggung-nyinggung keluarganya, terus mungkin dia nggak suka makanya
10
marah.” (“I guessed that it means „angry‟. I looked at his expression and the
sentences before, when I asked about his family. It seemed that he didn‟t like it, so
he became angry.”) Here, Shin used visual mode in learning the word „seethe‟ and
the focus is on character‟s expression.
Visual aid with the focus on event. This focus was used when the
characters were engaged in some actions of doing something that were illustrated
in the picture and used by the participants to learn the unknown words. The
example was when Della learned the word „wafts‟. She said:
“Aku guessing dari kalimatnya, mikir aja sih kan „Steam wafts lazily from the cup,
and I blow gently on it.‟ Terus dari situ udah mikir, oh apa ini mengepul ya
soalnya kan „steam‟, „steam‟ biasanya mengepul. Terus itu ternyata gambar
selanjutnya memperjelas, itu si Shiro lagi megang cup ramen. Dan itu ada uap-uap
yang lagi mengepul di situ.”
(“I guessed from the sentence, „Steam wafts lazily from the cup, and I blow gently
on it.‟ From that sentence I think that „wafts‟ means ‘mengepul’, because there‟s a
word „steam‟. After that there‟s a picture that made the portrayal clear; Shiro,
holding a cup of ramen, and the steam was wafting from it.”) From this statement,
it could be concluded that Della used visual mode in learning the word „wafts‟,
which was the steam from Shiro‟s cup ramen when he was holding it.
From the examples above, it can be seen that visual element could be used
in vocabulary learning through visual novel. The result corresponds with
Pressley‟s (1977) theory, that if a text is presented with related pictures, it will
leave more profound print in learner‟s mind. It is also in line with Underwood
(1989) who suggests that pictures are remembered better than texts, and texts are
remembered better if they are strongly related with the pictures which are
11
presented. Therefore, the role of visual in visual novel is not only to present words
in contextual picture, but also to make the new vocabulary easily understood by
the audience.
It is possible that two or three focuses are used in learning the same word,
depending on what pictures which appear in the contexts. Sometimes, another
element was also used to aid the participants in learning, however, two elements
coming up together could show that multimodal learning can be found in visual
novel, which will be discussed later.
Textual
Textual element takes form in text. It is the most important attribute in
visual novel because audience grasps the whole information of the story through
it. For example, Shin used the aid of written element in 15 unknown words, Yesha
in 23 words, and Della in 10 words (see Appendix A). Textual mode could be
categorized into two focuses; focus on form and focus on story.
Textual mode with the focus on form. This focus was used by the
participants through identifying clues from words, clause, and/or sentences that
surrounded the unfamiliar words. For example, all participants learned the word
„throng‟ as ‘kumpulan’ or ‘kerumunan’. The participants that used the focus on
form in learning the word „throng‟ were Shin and Della, while Yesha used both
focuses. Shin said, “Ini ada „throng‟. „Throng of students‟. Guessingnya sih kirakira-nya kayak „kerumunan‟ gitu, terus tadi buka kamus, iya bener artinya
„kerumunan‟.”
(“„Throng‟. „Throng of students‟. I think it‟s more like
12
‘kerumunan’. I also checked on the dictionary, and it‟s correct.”) The other
participant, Della, said, “Di sini ada „merging with the throng of students‟. Jadi
„kumpulan‟” (“There‟s a sentence „merging with the throng of students‟, so I
think it means ‘kumpulan’.”)
Both of them said that they learned the word „throng‟ through the sentence
„merging with the throng of students‟. From these examples, they showed how
words surrounding the unknown word could help them in learning the meaning
even though there was no visual or aural aid accompanying the textual mode.
Textual mode with the focus on story. This focus was used when the
participants recalled the story illustrated by the text to aid them in understanding
unknown vocabulary. The example was when Shin learned the meaning of
„brash‟, she said:
“Dari ceritanya, kan Alistair ni ngajak gelut kan, masak dia nyampah terus…
ngerebut item, padahal kan harusnya itu punyaku, terus kan juga aku yang kerja
keras. Terus kan aku bilang, aku mau „revenge‟. Terus si Fiona-nya ini takut kalau
aku mau ngelakuin „something stupid‟ gitu.”
(“I recalled the story, so Alistair was trying to pick a fight with me, he killed the
enemy that I was fighting to... and he ended up getting my reward item. After that,
my character said, „I want revenge‟. Then Fiona‟s afraid that I might do
„something stupid‟.”) Here, unlike in the previous focus that the clue is gotten
from the form of sentence, Shin recalled a part of the story that gave her clues to
learn the meaning of „brash‟.
13
Textual mode with the focus on form and story. It is also possible that
the two focuses are used in learning the same word. The example was when Della
learned the meaning of „PvP‟. During the interview, she said:
“„PvP‟. Guessing dari gambar, dari konteks. Soalnya kan awalnya si Alistair
ngerebut batunya Rui, terus mau duel, nah PvP itu aku nebak „Player versus
Player‟. Terus ada kata „match‟ juga, jadi itu kan mesti „gelut‟.”
(“„PvP‟. I guessed from the picture and context. At the beginning, Alistair stole
Rui‟s stone and they wanted a duel, so I guessed PvP means „Player versus
Player‟. There‟s a word „match‟, too, so it must be something related to „battle‟.”
In conclusion, Della used both focuses in learning the word „PvP‟; the story
illustration (that Alistair stole Rui‟s stone leading to a duel) and the sentence “The
PvP match you participated in with player Alistair has been cancelled”.
These focuses were in line with Kintsch (1988) suggestion, that “learning
with text consists of the construction of two interconnected mental representations
(i.e., a textbase and a situation model)” (cited in Zarei & Gilanian, 2013). The fact
that written mode was used by the three participants in most of the learning
process could prove the importance of text mode in visual novel, thus may
accompany the other elements or stand alone to be used as an aid in understanding
unfamiliar words.
Aural
Aural mode, the last multimodal element, is anything in multimedia which
can be heard by the ears as the audio sensory perception. In visual novels, usually
it can be found in characters‟ voices, background music, and sound effects. In this
visual novel, aural mode could be found through background music and sound
14
effects. Thus, these two became the focuses from aural mode. The only participant
that used aural mode was Yesha. She used it in 2 words (see Appendix A).
Aural mode with the focus on background music. This focus was used
when the participants focused on the music while playing and used it in learning
the meaning of unknown words. Background music may change depending on the
situations, moods, and some places and characters have their own music theme.
Nonetheless, Yesha did not use this focus at all. Instead, she used the other focus;
focus on sound effects.
Aural mode with the focus on sound effect. This focus was used when
the participants focused on the sound effects that appeared during the emergence
of unknown words. The example was when Yesha learned the meaning of „dash
off‟. Yesha said:
“„Dash off‟ itu aku guessing-nya lari pergi. Ada kata-kata ini, „I grab my books
and dash off to class.‟ Dari konteks kalimatnya aja, jadi kayak dia ambil buku
terus pergi. Terus kan ada suara bell masuk nya juga kan itu, jadi dia pergi buat ke
kelas.”
(“I think „dash off‟ is similar to ‘lari pergi’. There‟s a sentence „I grab my books
and dash off to class.‟ From the context I got the information that she took her
books then went off. Then there‟s the sound of the bell ringing, so she went to
attend the next class.”)
Based on the previous studies, it is believed that multimedia presentations
including background music improve more learning than in silent condition
(Groot, 2006; Thiessen & Saffran, 2009, cited in Phan & Cheon, 2013). However,
from the interviews with the three participants, it could be concluded that aural
15
mode in visual novel was not as helpful for the participants in understanding
unknown vocabulary. Yesha explained:
“Kalau suara, menurutku kok paling susah buat diandalin, karena kan kalau di
game ini cuma ada music latar sama efek suara, itu kan ya nggak pasti keluar pas
vocab yang nggak ngerti itu muncul kan. Terus kita interpretasinya juga tidak akan
sejelas gambar sama teks, kayak kalau misalkan ceritanya sedih musiknya juga
sedih, tapi mungkin buat aku sih musiknya bisa aja nggak sedih-sedih amat, jadi
kan belum tentu exact music-nya itu bisa menggambarkan tentang situasi yang
sebenernya.”
(“I think aural mode cannot be used as much, because in this game there are
background music and sound effects only, and they do not always appear when
the unknown vocabulary appears. We also cannot interpret audio element as clear
as visual and written elements, as example when the story‟s mood is sad then
maybe the music is also played in sad tone, but maybe for me the music is just
plain, so its ability to portray the exact situation is quite uncertain.”) From this
statement it showed that aural mode offers limited aids which resulted in
participant‟s limited understanding.
Shin also said, “Mungkin kalo misalnya ada suara orangnya ngomong
mungkin bisa lebih kebantu dari tone-nya dia.” (“Maybe, if there‟s any character‟s
voice, it can aid us through the tone.”) In this case, there are other visual novels
which offer more aural modes including character‟s voices, which can be used in
learning vocabulary. However, when asked whether she would like to play a
visual novel that does not provide any aural mode, Della said, “Sebenernya nggak
asik sih, soalnya suara juga mendukung mood, tapi untuk vocabulary nggak
terlalu crucial.” (“Actually I think that it‟s not fun, because the audio also boosts
the mood, but in vocabulary learning it‟s not very crucial.”) It shows that even
though she thought that audio element could not be used significantly in
16
vocabulary learning, she thought that audio element enhanced the mood and made
the visual novel more enjoyable.
Multimodal Elements in One Word Learning
Multimodal learning (Massaro, 2012) happens when two or three
representations appear in such context at the time the unknown word appears that
help the audience in learning the meaning. How each mode was used by the
participants in their vocabulary learning has been discussed, and this section
discusses how the three modes; visual, textual and aural were used in learning the
same word.
An example of how participant used the three modes in learning a word
was from Yesha, when she learned the meaning of „echoing boom‟. She said:
“„Echoing boom‟ itu kayak suara dribble-an dari basket, boom boom boom… Kan
tadi juga ada scene pas Derek main basket, suasananya keliatan dari gambarnya
tuh lagi di gym, terus ada suara boom boom boom dari main bola basketnya
juga… Kan ceweknya ini kan ke gym, terus kalimatnya „I can hear shouts and
echoing boom‟, jadi guess dari situ.”
(“‟Echoing boom‟ is like the sound of basket ball being dribbled, boom boom
boom... There was also a scene where Derek played basketball, the ambience was
shown from the picture that the characters were in a gym, then there was a sound
effect „boom boom boom‟ from the ball, too… The main character was going to
the gym, and there was a sentence „I can hear shouts and echoing boom‟, so I
guessed from those clues.”) Here, she mentioned „picture‟ as an instrument in
visual mode, „sound effect‟ as in aural mode, and „sentence‟ as in textual mode. It
means that Yesha used all of the multimedia elements in learning the unknown
vocabulary.
17
Error in Multimodal Learning Environment
Error here means when the audience misunderstand the word meaning due
to either limited clue that multimodes offer or audience‟s lack of understanding.
This error only occured in Yesha, with 7 errors (see Appendix A). For example,
Yesha learned the meaning of „spotty internet‟ as „a spot for accessing internet‟.
During the interview, she said:
“Spotty itu aku ingetnya kata „spot‟ dan spot itu kayak titik, satu tempat buat
kamu... Kayak istilahnya apa ya, koneksi wi-fi gitu lho, kalo aku guessing-nya sih
itu. Soalnya di sini kan ada gambar suasananya tuh lagi di ruangan computer gitu,
banyak komputernya. Nah spotty internet itu nggambarin tempatnya ini, spot buat
internetan.”
(“The word „spotty‟ made me think that it‟s like a „spot‟, so it is a place where
you… What‟s the term? It‟s like „wi-fi connection‟, in my opinion. It‟s because
there‟s a picture that portrayed the ambience, it‟s like a computer lab since there
were many computers. So the word „spotty internet‟ refers to the place, a place to
access the internet.”) From this statement, it was concluded that Yesha said that
because the word „spotty internet‟ appeared when the main character was shown
being in a computer lab. However, the meaning of „spotty internet‟ (from the
sentence “still, darn this school‟s spotty internet connection”) was more like the
performance of the internet connection which was „not always good‟. It could be
seen that multimodes could not always promising for true and direct meaning as
in dictionary, because multimodes here only facilitating clues to be used by the
participants. Therefore, the meaning of the unknown words resulted from the
participants‟ own thinking process.
18
Vocabulary Learning in Visual Novel
Visual novel can be considered as one of multimedia that is rich in
vocabulary due to its written element, and has visual and aural elements to help
portray the story. The advantage of learning vocabulary through visual novels
besides it enhances incidental learning and has multimodal elements is that its onclick setting. It means that the next text, followed by related pictures and sounds
appear only after the audience clicked the application. Therefore, the audience can
focus on the current event and grasp the meaning of the sentences, thus improve
vocabulary learning. This advantage does not correspond in other media such as
novels which have lots of text that may distract the reader‟s focus, and videos that
keep on going except they are being paused by the audience.
However, there is also ineffectiveness in visual novels regarding their role
as multimedia. Visual representations in visual novels are quite limited because
the pictures only change from one picture to another – unlike videos that provide
motions. Moreover, unlike visual element, aural element does not have crucial
role in visual novel. Therefore, some other visual novels might not provide any
audio at all. Just like its name; it is presented visually (as in visual) and textually
(as in novel), and aural is merely optional – it is still called „visual novel‟ even
when it does not have any aural element in it. The participants also said that aural
mode (in the form of background music and sound effects only) was the least
useful mode in learning vocabulary. There were two possibilities; the first was
because the sound effects and background music did not always appear where the
unknown word took place, and the second was because aural mode is more
19
intangible than the other modes. Nevertheless, music and sound effects in
multimedia might make the learning process become more enjoyable to learners,
thus incidental learning may happen (Weiner, 1990).
CONCLUSION
This research was conducted to see the types of multimodal elements in
visual novel that can be used by learners in learning English vocabulary. Based on
the data gathered through SRP method, it could be seen that every time unknown
word appeared, participants could make use of multimodal elements in the visual
novel, which resulted in incidental learning. Textual mode in visual novel had the
same nature as in general novel, and could be used to learn vocabulary meaning
through sentence form and story plot. Aural mode in this visual novel could only
be found in background music and sound effects, which were quite intangible to
be used in understanding unknown words, and did not always appear when the
unknown words appeared. Meanwhile visual mode could help in portraying the
scene (time and places), characters‟ expressions, and events, and used by the
participants in seeing the contexts – an advantage of visual novel compared to
common novel which only provides text. Although multimodes provide clues for
the participants to be used in learning unknown words, sometimes the meaning
that the participants got were not in direct meaning, but rather in close meaning
(not the exact synonym word, but they understand the meaning) or even resulted
in error.
20
The significance of the study was to give an insight that video games
could give meaningful purpose besides for fun; to acquire English vocabularies
through incidental learning. It was done due to the claim that video game could
increase gamers‟ English skill, although many people often see video games as a
pleasure provider only, not as a tool to provide learning environment. The
findings of the study might motivate people to use visual novel as a medium to
learn English vocabulary.
However, this research was still far from perfect due to some limitations,
which could be put into considerations for further research. There were only 3
participants that were involved in this research, so the result that visual novel can
be used as a medium in learning vocabulary might not be generalizable in
different context. Another problem was in getting the suitable visual novel to be
used. The visual novel that was used in this research, “RE:Alistair++” only
provided background music and sound effect as the aural mode, which could not
be used by the participants well in understanding the unknown vocabulary. It
would be better if the visual novel also provides characters‟ voices so it can be
used by learners more in understanding the tone and pronunciation. The
vocabulary level also has to be adjusted with the participants. If it is too easy, then
there may be no vocabulary learning at all. The content of the visual novel also
has to be considered. Sometimes, visual novels contain mature contents, such as
sexual and violence. Considering all of that, getting a visual novel that suits well
for the research is not easy. Therefore, using other types of video games or even
21
other multimedia can be an option to conduct a further research on vocabulary
learning in multimodal environment.
ACKNOWLEDGEMENT
First, I would like to thank God for giving me strength throughout
completing this thesis. I would also like to express my gratitude to my research
supervisor, Mr. Toar who always guided and gave me solutions, and my research
examiner, Prof. Astika who was willing to give me suggestions and corrections.
Without their assistance, this paper would not be as successful. My gratitude also
goes to my family and friends for their encouragement every time I feel down. I
thank my brother too, for lending me his laptop during my thesis writing because
mine was broken. For Hasenvan, thank you for suggesting me to use the visual
novel. I did not know this video game would become the topic of my thesis. Last
but important, I want to thank my participants who gave me their time and efforts
during data collecting process. Without their contribution, this paper would have
never been done.
REFERENCES
Ainsworth, S., & Van Labeke, N. (2002). Using a Multi-representational Design
Framework to Develop and Evaluate a Dynamic Simulation Environment.
Retrieved
May
1,
2015,
from
http://www.nvl.calques3d.org/publications/2002.DynVis.Framework.pdf
Deci, E. L., Vallerand, R. J., Pelletier, L. J., & Ryan, R. M. (1991). Motivation
and Education: The Self Determination Perspective. Educational
Psychologist, 26 (3 & 4), 325-346.
22
deHaan, J., Reed, W. M., & Kuwada, K. (2010). The Effect of Interactivity with
a Music Video Game on Second Language Vocabulary Recall. Language
Learning & Technology, 14, 1094-3501.
Gass, S. M., & Mackey, A. (2000). Stimulated Recall Methodology in Second
Language Research. Retrieved May 1, 2015, from
http://www.cc.kyoto-su.ac.jp/information/tesl-ej/ej16/r18.html
Gu, P. Y. (2003). Vocabulary Learning in a Second Language: Person, Task,
Context and Strategies. Retrieved October 5, 2014, from Teaching English
as a Second or Foreign Language Web site:
http://www.tesl-ej.org/ej26/a4.html
Huyen, N. T. T., & Nga, K. T. T. (2003). Learning Vocabulary through Games:
The Effectiveness of Learning Vocabulary Through Games. Retrieved
September 21, 2014, from Asian EFL Journal Web site:
http://www.asian-efl-journal.com/dec_03_vn.pdf
Kim, D., & Gilman, D. A. (2008). Effects of Text, Audio, and Graphic Aids in
Multimedia Instruction for Vocabulary Learning. Educational Technology
& Society, 11 (3), 114-126.
Kozma, R. B. (1991). Learning with Media. Review of Educational Research,
61(2), 179-212.
Leu, D. J. (2002). Internet Workshop: Making Time for Literacy. The Reading
Teacher, 55(5), 466-72.
Massaro, D. W. (2012). Multimodal Learning. Retrieved November 14, 2014,
from https://wss.apan.org/jko/mls/Learning%20Content/Massaro%20%20Multimodal%20Learning%20(Encyclopedia%20of%20the%20Scienc
es%20of%20Learning).PDF
Mayer, R. E. (2003). Elements of a Science of E-learning. Journal of Educational
Computing Research, 29(3), 297-313.
Meihami, H., Meihami, B., & Varmaghani, Z. (2013). CALL in the Form of
Simulation Games: Teaching English Vocabulary and Pronunciation
through Sims. International Letters of Social and Humanistic Sciences, 8,
2300-2697.
Moreno, R. (2002). Who Learns Best with Multiple Representations? Cognitive
Theory Implications for Individual Differences in Multimedia Learning.
Retrieved
May
1,
2015,
from
http://www.editlib.org/p/10238/proceeding_10238.pdf
Nation, P. (2002). Best Practice in Vocabulary Teaching and Learning. J. C.
23
Richards & W. A. Renandya (Eds.), Methodology in Language Teaching:
an Anthology of Current Practice (pp.267-272). London: Cambridge.
Nation, P. (2005). Teaching Vocabulary. Retrieved November 24, 2014, from
Asian EFL Journal Web site:
http://www.asian-efl-journal.com/sept_05_pn.pdf
Phan, V., & Cheon, J. (2013). Multimedia Learning: Effects of Background Music
and Sound Effects on Space Science Concept Learning. Retrieved March
29, 2015, from www.aect.org/pdf/proceedings13/2013/13_23.pdf
Pressley, M. (1977). Imagery and Children's Learning: Putting the Pictures in
Developmental Perspective. Review of Educational Research, 47(4), 585622.
Sankey, M., Birch, M., & Gardiner, M. (2010). Engaging Students through
Multimodal Learning Environments: the Journey Continues. Retrieved
November 14, 2014, from
http://ascilite.org.au/conferences/sydney10/procs/Sankey-full.pdf
Thornbury, S. (2002). How to Teach Vocabulary.
England: Pearson Education Limited.
Turgut, Y., & İrgin, P. (2009). Students’ Experiences of Computer Games and
ELL: from Phenomenological Perspective. Retrieved September 9, 2014,
from http://dbe.metu.edu.tr/convention/proceedingsweb/Games.pdf
Underwood, J. (1989). Hyper Card and Interactive Video. CALICO Journal, 6 (3),
7-20.
Weiner, B. (1990). History of Motivational Research in Education. Journal of
Educational Psychology, 82 , 616-622.
Zarei, A. A., & Gilanian, M. (2013). The Effect of Multimedia Modes on L2
Vocabulary Learning. International Journal of Management and
Humanity Sciences., 2 (S), 1011-1020.
24
APPENDIX A
Table of Participants’ Unknown Words, Multimodals Used and Actions
*: error in processing meaning
Participants
Shin
Yesha
Words
Multimodals
Used
AP & MP
forfeited
huddle
textual, visual
Textual
Textual
recluse
menial
brash
throng
shallow
postulated
bracket
noutorious
pristine
prudent
text, visual
Textual
textual
textual
textual
textual
textual
textual
textual
textual
seethe
imposing
textual, visual
textual
Actions
focus on form (textual), focus
on event (visual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual), focus on character's
expression (visual)
focus on form (textual)
focus on story (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual)
blessed stone
spotty
internet*
visual
focus on event (visual)
visual
forfeited
textual, visual
dash off
scribble
menial
textual, aural
textual
textual
smirking
brash
rotten*
textual, visual
textual
textual
focus on place (visual)
focus on story & form
(textual), focus on place
(visual)
focus on form (textual), focus
on sound effects (aural)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual)
focus on form (textual)
25
Della
echoing
boom
something of
a stretch*
merging
throng
heads off
textual, visual,
aural
drag*
postulated
bracket
textual
textual
textual
sighs*
textual, visual
chilling
textual, visual
chuckles
wipes off
glance
sensible*
picky*
visual
textual
textual
textual
textual
waft
page
textual, visual
textual
textual
textual
textual
textual
menial
textual
PvP
textual, visual
brash
throng
postulated
social butterfly
textual
textual
textual
textual
sketchy
spacebook
savvy
textual
textual
textual
waft
textual, visual
26
focus on form (textual), focus
on place (visual), focus on
sound effects (aural)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual)
focus on form (textual)
focus on form (textual)
focus on story (textual), focus
on character's expression
(visual)
focus on form (textual), focus
on place (visual)
focus on event & character's
expression (visual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form (textual), focus
on event (visual)
focus on form (textual)
focus on form (textual)
focus on story & form
(textual), focus on event
(visual)
focus on story & form
(textual)
focus on form (textual)
focus on form (textual)
focus on form (textual)
focus on form and story
(textual)
focus on form (textual)
focus on form (textual)
focus on form (textual), focus
on event (visual)
APPENDIX B
Transcription of the Interviews with Participants
Transcription 1
W: Wenda
S: Shin
W: Selamat sore.
S: Sore.
W: Oke tadi kan sudah main visual novel, Alistair. Ada kata atau vocab yang
belum pernah tau atau baru dipelajari di game ini?
S: Iya, ada beberapa.
W: Kebanyakan guessing atau pakai kamus?
S: Kalau kata-kata yang bisa nebak artinya kira-kira apa gitu aku nebak dulu, terus
buka kamus buat mastiin artinya. Soalnya kalau nggak gitu nanti takutnya salah
milih options.
W: Oke. Tadi katanya nggak tau yang „AP‟ sama „MP‟. Tapi ada gambaran nggak
„AP‟ sama „MP‟ itu apa?
S: Karena tadi tulisan „AP‟ sama „MP‟-nya pakai huruf besar semua, jadi…
mungkin itu… ini sebenernya guessing ya, tapi kalo menurutku ini…. Vocab
sekitar game-game gitu. Soalnya kan ini juga lagi main game, main game online
gitu. Kalo misalnya attack, atau defense gitu kan pakai „ATK‟, „DEF‟ kan huruf
besar semua, kalo ini kan juga tadi „AP‟ sama „MP‟ pake huruf besar semua, jadi
ya itu mungkin ya sodara-sodaranya „ATK‟, „DEF‟ gitu.
W: Jadi guessing-nya itu clue-nya dari bentuk kalimat?
S: Iya, bentuk kalimat sama bentu katanya sendiri itu.
W: Taunya itu lagi main game dari mana?
S: Itu screen-nya lagi main game.
W: Oke. Nemu kata apa lagi yang susah?
S: Ini bacanya apa ya, „forfeited‟.
W: Untuk tau itu artinya, gimana?
27
S: Ini sih otomatis guessing, mungkin artinya kayak „mundur‟ gitu, mundur dari
permainan, soalnya kan kalau ngeliat dari kalimat sebelumnya internetnya nge-lag
gitu, terus kalimat sesudahnya ini juga kayak ngomongin efek dari kalimat yang
tadi sebelumnya itu. Jadi aku nyimpulin kalo gara-gara internet-nya nge-lag,
makanya terpaksa „mundur‟, karena itu juga kan pa