Analysis of the Data

137 speaking productions that were not well-represented in the materials. Such as, after what is it, diagram, after serious of dialogues, there is no follow up activity. I Ie 6; page 266 Furthermore, the following interview part indicated that although some units might not have good integration of English four skills, the integration can be improved when the designed materials are implemented in the classroom. However, the second evaluator concluded that overal units have integrated English four skills. Yeah, yeah, indeed. Absolutely. Because I can see, I think, not all units have the four skills, but later on, in the class, maybe like through conversation and discussion with the students, you can also build their confidence to speak, for example and also to listen to your explanation, yeah. Although maybe there is no listening in certain units, maybe, ya. But in the class, the listening can be from the teacher. J Ie 6; page 267 The third evaluator showed her agreement on the good integration of English skills by giving details on the skills implemented in units. It can be seen from the following interview part. I think, the integration of listening, speaking, reading, and writing are quite balanced because the meeting and this meeting will focus on reading, the next reading will focus on listening, and etc. And then, in some parts, I found that first of all, you provide the listening passage, after that the students have to write something to make the summary so that’s good for the students. And then in reading part, yeah, thats’s good also. K Ie 4; page 269 Moreover, the forth evaluator stated that the skills are well implemented without giving any other suggestions. It can be seen from the following interview part. I have no complaint on it. Things are just awesome. The four strands are well implemented. L Ie 4; page 270 PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI 138 As the conclusion of the discussion above, the English skills are already well integrated. Additionally, there should be paid more attention on the speaking activities as one of the evaluators noticed that there should be more speaking activities in particular units. d Theme 4: The language level is already appropriate, but need to be adjusted. As the level of the students is A1 CEFR, the language in the designed materials should meet their language level. All of the evaluators agreed that the language is already appropriate, although in some parts, the language need to be adjusted. The following interview part shows the first evaluator’s assessment toward the language level. Language type is now more simplified and more adjusted, well- adjusted to students ability. And what is it, probably my suggestions are in the vocabulary building parts section, probably some definitions of words are much more confusing than the word itself. I Ie 9; page 266 Regarding to the use of vocabulary items that were used to explain some difficult words, the evaluator prefered to use Indonesian terms. Besides, the second evaluator agreed that the language is already appropriate with the students’ level. It can be seen from the following interview part. But what I see in your new design, I can see that, yeah, you have already made the language appropriate for the students and also the students can understand and then also related to the... I think, I gave you a comment on reading, reading or listening? The text.. J Ie 7; page 268 Furthermore, the third evaluator seemed to have the same opinion with the first evaluator. It was about dealing with the vocabulary items that were used to 139 explain some other vocabulary items. From the following interview part, it can be seen that the evaluator suggested the students to check them in the dictionary. OK. I think the grammar like appropriate for them, but in some texts, maybe the vocabulary will be hard for them. Because in your design, you use some new vocabularies or new vocabularies that are rarely used by them. But that’s okay, because they can check from the dictionary, right? K Ie 6; page 269 Moreover, the forth evaluator also showed his agreement that the language used in the designed materials already met the students’ competence. Regarding to some vocabulary items that were seen as challenging for them, the evaluator suggested that the researcher should explain more in classroom. It can be seen from the following interview part. Most of the diction presented will fit with the learners’ competence. They could negotiate the meaning of a difficult concept in class. L Ie 5; page 270 Regardless of getting some suggestions to simplify some vocabulary items, it can be concluded that the language of the designed materials is, overall, already appropriate. Therefore, the students would later find it relatively easy to comprehend the contents of the materials through the suitable language level utilized in the designed materials. e Theme 5: The materials are already appropriate to be implemented, but need to be revised. Overall, all of the evaluators agreed that the designed materials are already appropriate to be implemented. However, some parts need to be revised based on the suggestions they gave in the field notes later on. The following interview PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI 140 parts contain the agreement of the first evaluator that the designed materials are already appropriate. Oh, very very very appropriate. Level five. I Ie 10; page 266 Furthermore, the second evaluator also agreed that the designed materials hold the appropriateness to be implemented. She mentioned the percentage of the appropriateness. The rest of the percentage left was for the recordings. It can be seen from the following interview part. Yeah. It is already... It is ready to be implemented to the students. It is ready to be implemented like 80. 20 is for the listening and the recording. J Ie 10 – Ie 11; page 268 The third evaluator showed her agreement toward the appropriateness of the designed materials. She, also, acclaimed the detailed components in the designed materials, such as the design, layout, materials, etc. It can be seen from the following interview part. I think your materials are already ready to implemented to your students because the design, the layout, materials, and everything inside your design is very well prepared. K Ie 7; page 269 Lastly, the forth evaluator said that the designed materials have already been ready to be implemented. He suggested to pay attention to the comments provided in his field notes. The following interview part shows his opinion. It’s ready to use and launch. Please consider some comments as my field notes. I have to appreciate the way you create such a good layout. L Ie 6; page 270 By looking at the discussion above, it can be concluded that the designed materials are already appropriate to be implemented in the classroom. Considering the comments and suggestions delivered by the evaluators, the PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI 141 designed materials would be revised, so that the students would use the better version of the designed materials. 2 Questionnaires Analysis As a part of the preliminary field testing, the questionnaires were distributed to four evaluators. There are 54 questions covering five topics: 1 subject content, 2 skills, 3 sub-skills, 4 layout physical make-up, and 5 practical considerations. The raw data of the questionnaires can be seen in Appendix C. The first topic is subject and content. In dealing with subject and content of the designed materials, there are 10 questions provided in the questionnaires. Here are the criteria of the total mean score of this topic: 10.00 – 18.00 18.01 – 26.00 26.01 – 34.00 34.01 – 42.00

42.01 – 50.00

Strongly disagree Disagree Undecided Agree Strongly agree The total mean score of this topic was 42.50. It means that all of the evaluators strongly agreed that the designed materials have already had appropriate subject and content. Although the results of the questionnaires showed that the subject and content were already appropriate, it needed to be adjusted with the results of the interview and field notes. English skills are as the second topic of the questionnaires. The skills cover listening, speaking, reading, and writing skill. A total of the questions dealing with skills is 12. The followings are the criteria of the total mean score of the topic: PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI 142 12.00 – 21.60 21.61 – 31.20 31.21 – 40.80

40.81 – 50.40

50.41 – 60.00 Strongly disagree Disagree Undecided Agree Strongly agree

48.50 was the total mean score of this topic. The result represented the

criterion of agree. It means that all of the evaluators agreed that the designed materials have integrated English skills covering listening, speaking, reading, and writing. The integration of those English four skills was balanced by the roles of exercises as well as media that support the implementation of the skills. Moreover, the results of this questionnaires would be adjusted by the results of the interview and field notes. The third topic of the questionnaires deals with sub-skills. The sub-skills provided in the questionnaires cover vocabulary, grammar, and pronunciation. There are 13 questions provided in the questionnaires. Here are the criteria of the total mean score: 13.00 – 23.40 23.41 – 33.80 33.81 – 44.20

44.21 – 54.60

54.61 – 65.00 Strongly disagree Disagree Undecided Agree Strongly agree The total mean score of this topic was 49.00. The result is indicated to be in the criterion of agree. It means that, overall, the designed materials have already been appropriate in integrating the learning of vocabulary, grammar, and pronunciation in the designed materials. 143 As the forth topic of the questionnaires, layout and physical make-up deals with face validity of the designed materials. There are 7 questions provided. The followings are the criteria of the total mean score. 7.00 – 12.60 12.61 – 18.20 18.21 – 23.80 23.81 – 29.40

29.41 – 35.00

Strongly disagree Disagree Undecided Agree Strongly agree The total mean score of this topic was 31.25. The result indicates the criterion of strongly agree. All of the evaluators agreed that the designed materials contain appropriate layout and physical make-up. The last topic of the questionnaires is about practical considerations. It deals with the practicality when the designed materials are implemented in the classroom. Furthermore, there are 12 questions about practical considerations. The followings are the criteria of the total mean score: 12.00 – 21.60 21.61 – 31.20 31.21 – 40.80

40.81 – 50.40

50.41 – 60.00 Strongly disagree Disagree Undecided Agree Strongly agree

49.00 was the total mean score of this topic. All of the evaluators agreed

that the designed materials already contain appropriate practical considerations. One of the considerations is that the students will be able to use the designed materials outside the classroom to support their independent learning. 3 Field Notes Analysis Field notes from the evaluators are the third data that were used to verify the acceptability of the designed materials. After compiling all of the field notes, 144 the data were analyzed qualitatively and generated into five themes, which can be seen in Appendix B on page 278-280. Those five themes are there should be revision to the 1 activities, 2 instructions, 3 contents, 4 contents, and 5 diction. In the following discussion, the raw data that were included in the explanation only represent the notes from each evaluator. Furthermore, the raw data of the field notes can be seen in Appendix C. a Theme 1: There should be revision to the activites The first theme is related to the learning activities provided in the designed materials. Some evaluators suggested that some activities need to be adjusted. Adjusting the activities can be by adding any other additional activities, omitting the existing activities, or adjusting the appropriateness of the activities to the students’ English competence. Some suggestions can be seen from the following notes: [Evaluator 1] “Let’s Prepare in Unit 7: Give full dialogue about how to explain rundown for particular schedule so that the students can refer to the dialogue. ” [Evaluator 4] “Unit 5: Later, in practice, you must make sure which would become the focus: describing foods or describing future plans. Two big agendas. Describing food would consist of letting them know about adjective order description, size, age, color, materials, origin. ” b Theme 2: There should be revision to the grammar Grammar is the second theme that is mostly considered by the evaluators. Some evaluators noticed that the grammatical mistakes do not only appear in particular units, but also mostly in units of the designed materials. However, the grammarical mistakes were still considered as minor mistakes. The notes from the evaluators can be seen below. PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI 145 [Evaluator 1] “Let’s Prepare in Unit 7: “will be dressed” → why passive? Aren’t your students confused?” [Evaluator 2] “All units: Mind the grammar. There are few mistakes. ” [Evaluator 3] “Unit 7: Grammar mistakes.” [Evaluator 4] “Let’s Reflect in Unit 3: Check the consistency of the use of tenses. Check again the grammar. ” c Theme 3: There should be revision to the instructions The third theme is the instructions. The instructions bring an important role as they contain imperative sentences for the students to do the tasks. In order to have better version of the instructions, most evaluators suggested to revise some instructions in terms of the language used. The following answers represent their suggestions. [Evaluator 2] “All units: Be consistent in creating the instructions. For example, instructions in listening section. ” [Evaluator 3] “Let’s Do It in Unit 2: Change “Arrange the following random paragraphs into a good order ” into “Arrange the following random sentences into a good paragraph ” [Evaluator 4] “Let’s Do It in Unit 2: Should it be explain or arrange? ” d Theme 4: There should be revision to the contents As the forth theme, the contents of the designed materials were seen as parts that need to be revised. All of the evaluators paid their attention to the contents particularly about the contents of both the input or the exercises. Their suggestions can be seen from the following answers. [Evaluator 1] “Let’s Prepare in Unit 2: “There are eleven ceremonies...” → not comprehension question.” [Evaluator 2] “All units: Put source of pictures you use in the materials. ” [Evaluator 3] “Let’s Learn in Unit 6: Edit the information of dates.” [Evaluator 4] “Let’s Learn in Unit 5: The use of will is not the same with be going to, somehow. Check again the difference. ”