Evaluation and Revision Process of Developing the Iconic Model
96
mean score was 1.28. It indicated that the materials were good and acceptable for
the tenth grade students of Senior High Schools. The validators agreed that the materials in the application were interesting, easy to understand, and could help
students practice listening. They also considered that FLAP was practical and flexible because it could be accessed through mobile devices, such as smartphones
and tablets. This is in line with the principles of m-learning proposed by Elias 2011 number 1 equitable use and 2 flexible use. In addition, the experts also
agreed that the instructions were easy to follow. It corresponds to the principle number 4 perceptible information and 6 low physical and technical effort.
However, there were some aspects which still need some attention. Statement number 2 scored 0 and it was categorized as fair. It was about the novelty
of the materials. Therefore, some revision was made related to the selection of materials.
The score of the last statement i.e. 0.5 indicated that the guidance of the teacher tutor was less required. It showed that FLAP could facilitate students in
practicing listening independently. From the results of the open-ended questions in the questionniare see Appendix 9, both material experts agreed that FLAP was an
innovation in English learning by optimizing the use of technology so that it could increase students’ motivation in learning English. Conceptually, FLAP can be used
as supplementary materials to improve students’ listening ability. The materials experts mentioned some strengths of FLAP. First, there are
various materials that can be used to practice listening. Next, FLAP is easy to operate. Besides, the size of the application is quite small, thus students do not need
long time to download and install the application in their mobile devices.
97 Furthermore, the simple design of the application enables the students to understand
the instruction in the application easily. In addition, the materials validator said that FLAP could encourage the
students to learn by using their mobile phones. Moreover, they would consider their phones as one of the learning tools. However, regarding with the materials, there
were some aspects needed to be paid attention to. First, the level of difficulty and complexity of the materials should be taken into account.
“...For listening aspect, there were some levels considered as low for Senior High School level. But, there were also some materials considered
as high for this level. I think it is necessary to adjust the materials. You should consider about the number of the vocabulary and also the native
speaker’s voice. You could see the National Examination as the
reference...” Appendix 11 The materials expert also stated that by using FLAP it would save the cost
and space. The school did not need to provide a language laboratory, if the students had had their own laboratory i.e. their mobile phones.
Second, the main important aspect was about the monitoring. To be successful in implementing this mobile learning model in the classroom, the teacher
needed to patiently and seriously monitor the students’ activities during the teaching
and learning process.
“...Perhaps, we have believed that our students are seriously listening using FLAP. In fact, they are just listening to musics or videos.
We don’t know it, right?...” Appendix11
Third, the teacher needed to give useful feedback after students used the application. When the students got difficulties in some aspects, teachers should
explain more or the students could ask their friends about their problems.
98 The materials expert also stated that the songs were interesting. The students
could enjoy learning English through songs. He suggested that it would be more meaningful if the songs were related to the specific materials that were being learnt.
“...The materials using songs were very interesting. However, you needed to find out the songs related to the materials. For example, I ever used the
song from The Purple “Soldier of Fortune” to tell about Narrative. We should find unfamiliar songs. Then, if we wanted to teach about Adjectives,
the adjectives in the songs should be omitted. If narrative, then the Verb 2s
were omitted....” Appendix 11 The second part was about the media aspects. The technique of analysis was
also the same with the materials validation. The results of the questionnaires were presented in the following table.
Table 4.11. The Descriptive Statistics of Media Experts ’ Opinion
No. Statements
V3 V4 Mean
Category
1 The elements which have the same
functions have the same colour. 1
2 1.5
Very High 2
The navigation in this application is consistent.
2 2
2 Very High
3 The use of icons, symbols, and
buttons is consistent. 1
2 1.5
Very High 4
The layout design is attractive. 1
-1 Fair
5 All contents can be presented in
one screen. 2
2 2
Very High 6
The font style and colour in this application is appropriate.
2 -1
0.5 High
7 Texts are legible.
1 2
1.5 Very High
8 Different elements have contrast
colours. 1
-1 Fair
9 Navigation logos and buttons are
repeated in every screen. 1
2 1.5
Very High 10
Every element is aligned parallel, either horizontally or vertically.
1 2
1.5 Very High
11 The
elements which
are interrelated are put closer.
1 1
1 High
12 The navigation in this application
is easy to understand. 1
2 1.5
Very High 13
The application has a high tolerance of error.
2 1
1.5 Very High
continued
99 continued
No. Statements
V3 V4 Mean
Category
14 The instruction in this application
is clear. 1
-1 Fair
15 The
audio quality
in this
application is good. 1
-2 -0.5
Low
Total Score 15.5
Good Mean
1.03
The mean score i.e. 1.03 indicated that the results were good. Overall, the result of the media validation showed that the application was good enough.
According to the interpretation, the revision about media aspects was optional. Both experts agreed that FLAP had a consistent and easy navigation. It is in line with the
principles of m-learning Elias, 2011, number 3 simple and intuitive and 6 low physical and technical effort. The result also showed that FLAP had a high tolerance
of error principle number 5. Because the instruction was clear, FLAP required an easy operation.
However, there were some aspects which needed some improvements, such as design, layout, colour, instruction, and audio. Thus, the researcher needed to
conduct some revisions toward the design and audios. The result of the open-ended questionnaire also showed the similar result. The validator suggested to improve
the quality of the audios.
“The compression of the audio is too small so the quality of the audio is not too good” Appendix 9
According to the interview, the media expert still questioned about the scoring and feedback. The application did not provide a data storing system which
could record the students’ scores. Thus, if the students used the application at home and did the exercises they could not know their difficulties and progress.
100
“... moreover about the data storage. If the students do the exercises at
home, their progress and difficulties can be monitored.... The next is about scoring
. If their score is low, teachers can know what their difficulties are.” Appendix 11
Besides, the quality of the audios was still poor. Some of the audios were not audible enough. Then, he also suggested to add some hints for the blanks or
missing lyrics in the exercises. It would help the students to guess the words. In conclusion, based on the questionnaire and the interview, the experts state
that FLAP is good and acceptable for the tenth grade students. They agree that FLAP can help students in learning English anytime and anywhere. However, there
were some parts of FLAP which required some revision.
Table 4.12. The Comments and Suggestions from the Experts No.
Aspects Comments
Suggestions
1. Design
Too rigid,
not really
attractive Use
other application
creator to design the layout 2.
Audio The compression of the
audio is too small, thus the quality of the sound is not
good To improve the audio
quality
3. Variation of
materials Lack of materials variety
There should be more than one materials in every unit.
4. Font
Too thick Do not bold some fonts and
lower the font size 5.
Scoring No scoring form
There should be a feature to check students’ scores
6. Resolution
Should be able to be used in many kinds of resolution in
different mobile devices The resolution should be
adjusted so that it can be used
in tablets
and smartphones
7. Manual
In the “About” section, there is no manual on how
to use the application There should be a manual
on how to operate the application
After getting much useful feedback from both materials and media experts, the first revision was done to improve the product. Some changes were done to
revise the application according to the experts’ suggestion. The design of the
101 background and layout was revised. Scoring was also attached in each exercise. The
description about the functions of each button was added. b.
Users Validation The users of this application were the tenth grade students. To meet the
students’ needs, it was essential to gather students’ opinion, suggestion, and feedback about the application. The validation by the users were done by
distributing questionnaires and conducting interviews. The questionnaires were divided into two stages: Preliminary Field Testing and Main Field Testing.
1 User Validation Preliminary Field Testing
The users validation in Preliminary Field Testing was done in Thursday, May 14th, 2015. There were ten students selected as the respondents in this stage.
They were asked to fill in the questionnaires. The questionnaire results were analyzed by using the interpretation scores
like those in the experts questionnaire analysis. The students stated that the materials in FLAP could help them practice listening. FLAP provided new and
interesting materials for them. Besides, the materials were easy to understand. They found it more enjoyable to learn English using FLAP.
Table 4.13. The Users’ Opinions toward the Materials Aspects
No. Statements
Mean Category
1. The materials in FLAP can help the students
practice listening. 1.3
Very High 2.
The materials in FLAP are interesting. 1.1
High 3
The materials in FLAP are new for the students. 1.4
Very High 4.
The materials in FLAP are easy to understand. 0.8
High 5.
Learning English using FLAP is more enjoyable.
1.1 High
It was also found that the division of menu in FLAP into Focus, Fun, and Final could facilitate them to learn easier. The materials which had been adjusted
102 with the syllabus also supported the students in learning. In addition, different types
of exercises i.e. true-false, multiple choice, and filling blanks, also beneficial for them. They also liked the Fun aspects, particularly songs. It can be seen from the
following data.
Table 4.14. The Users’ Opinions about the Parts of FLAP
No. Statements
Mean Category
1. The materials in FLAP are in line with the
syllabus. 0.7
High 2.
The exercises help the students in practicing listening.
1.1 High
3. The division of the activities in FLAP Focus,
Fun, and Final help the students learn listening easier.
1 High
4 The students can learn independently through
FLAP. 1.1
High 5.
FLAP can help the students increase their vocabulary.
1.1 High
6. Focus section in FLAP can ease the students
with listening materials. 1
High 7.
Fun section is interesting because it provides some enjotable activities.
1.3 Very High
8. Final Section gives opportunities for the
students to practice doing some listening exercises.
1.3 Very High
Based on the result of the open-ended questionnaires, the students stated that FLAP was very useful to help them practice listening. They also mentioned about some
advantages of FLAP. FLAP is easy to operate, practical, has a small size, and can be accessed without an internet connection. Besides, the materials are also
interesting, especial ly the songs. One of FLAP’s strength is its portability. It was in
line with the basic concept of mobile learning i.e. learning anywhere and anytime.
“It is portable. We can learn from our mobile phones, no need to open the
books.” Appendix 11
103 Related to listening comprehension, FLAP could help students develop their
listening skills as well as improving their vocabulary. Different topics, exercises, and activities could vary students’ experiences of listening activities.
“It is helpful. It can increase our vocabulary and improve our listening skills
.” Appendix 11, S19, S9 Nevertheless, there were some weaknesses of FLAP. The quality of the audio was
not really good. It was shown by the mean of -0.1 see Appendix 10. Moreover, most students said that the design and layout were not attractive. In addition, the
materials also needed to be improved and varied. It was necessary to provide links to websites or downloadable contents so that the materials can be up-to-date and
upgraded. “It is good. But, the number of the exercises is only 15. So, if all exercises
have been done, then it is finished. I think you need to add more.... Maybe you can add some games, or downloadable contents. So it can
automatically update. Then, we only need to simply download it.”
Appendix 11, S15 The results from the questionnaire in the preliminary field testing were used
to make the revision toward the application. The researcher made some consideration about some possible suggestion which could be done to revise FLAP.
After FLAP had been revised, it was then field tested. 2
User Validation Main Field Testing The users validation of the Main Field Testing was conducted on
Wednesday, June 13th, 2015. The questionnaires were distributed to 32 students of the tenth grade students in SMAN 1 Depok Sleman. They are asked to explore
FLAP and to try it at home. After that, they were asked to fill in the questionnaires to gather their general opinions and feedback about the application. The results are
presented in the table below.
104
Table 4.15. The Result of Users Validation Questionnaire in the Main Field Testing
No. Statements
N Total
Mean Category
1. The materials in FLAP can help
me practice listening. 32
42 1.3
Very High 2.
The materials in FLAP are interesting.
32 31
1 High
3. The materials in FLAP are new for
me. 32
25 0.8
High 4.
The materials in FLAP are easy to understand.
32 27
0.8 High
5. Learning English using FLAP is
more enjoyable. 32
27 0.8
High 6.
FLAP is really practical because it can be accessed through mobile
devices, such as handphone and tablet.
32 39
1.2 Very High
7. FLAP can be used as a learning
tool to practice listening, not only in the classroom, but also outside
the classroom. 32
38 1.1
High
8. The audio transcription helps me
understand the materials. 32
26 0.8
High 9.
The instructions in FLAP are easy to follow.
32 29
0.9 High
10. The materials in FLAP are in line
with the syllabus. 32
27 0.8
High 11.
The exercises multiple choice, true-false, and filling blanks help
me practice listening. 32
36 1.1
High
12. The division of the activities in
FLAP Focus, Fun, Final, and Forum helps me to learn listening
easier. 32
30 0.9
High
13. I can learn independently through
FLAP. 32
33 1
High 14.
FLAP can help increase my vocabulary.
32 30
0.9 High
15. Focus section in FLAP can ease
me with listening materials. 32
34 1
High 16.
Fun section is interesting because it
provides some
enjoyable activities.
32 35
1 High
continued
105 continued
No. Statements
N Total
Mean Category
17. Final Section gives opportunities
to practice doing some listening exercises.
32 37
1.1 High
18. Forum Section can facilitate me to
communicate with friends or teachers.
32 31
1 High
19. FLAP can vary the media to
practice listening. 32
33 1
High 20.
Teacher’s guidance is required in using FLAP.
32 13
0.4 High
21. The use of fonts, backgrounds, and
colours in each unit is consistent. 32
25 0.8
High 22.
The content is presented in one screen so it make the students
understand the materials easier. 32
26 0.8
High
23. The font size is appropriate so the
texts are legible. 32
25 0.8
High 24.
The design of FLAP is attractive. 32
-5 -0.1
Fair 25.
The font type and colour in FLAP is appropriate.
32 24
0.8 High
26. The navigation in FLAP is easy.
32 26
0.8 High
27. The quality of the audios in FLAP
is good. 32
22 0.7
High
Total 766
0,9 Good
According to the questionnaire result, it can be inferred that generally FLAP is good and acceptable to be used as one of supporting tools for listening in Senior
High Schools Grade X by the final score 0.9. Most users state that FLAP is practical
and can help them practice listening. One strength of FLAP is its flexibility because it can be accessed through mobile devices, such as smartphones and tablets.
Therefore, it enables students to learn by using FLAP anywhere and anytime. Most students stated that the teacher’s guidance was not really necessary. It
means that the students could learn independently by using FLAP. However, there
106 was an unsatisfying result related to design and layout. Many students still thought
that the layout was not attractive enough. Some revisions toward FLAP have been conducted, including the design
and some other parts. Initially there are only three main parts Focus, Fun, and Final. In the final version of FLAP, Forum is added to provide a means for students
to share something or to communicate with their friends or teachers when they find difficulties. However, the audio does not get any changes because of some
limitation of App Inventor. To get better quality of audios, the original files should be in MP3 format with a big file size. It is hard to be done since all the audios are
offline and packaged with the application. If the sizes of the audios are large, the application will not be able to be built through the App Inventor. It is important to
select possible revisions toward the product because the Android development program i.e. App Inventor as well as the smartphone has some limitations.