Evaluation Instrument Preliminary Field Testing

73 Respondents Sex Educational Background Teaching Experiences in Years Male Female S1 S2 S3 1-5 6-10 11-15 16-20 20 The English lecturer of ELESP Sanata Dharma University √ √ √ The English lecturer of ELESP Sanata Dharma University √ √ √ The researcher distributed the evaluation questionnaires which consisted of selected-response questions and open-ended questions. In the selected response questions, the evaluators were asked to put a tick in the points of agreement available about the topics, learning objectives, reading texts, writing tasks, the exercises, the instructions, the layout and the overall presentation of the designed materials. The open ended questions consisted of two questions: first the strengths and the weaknesses of the designed materials and second the feedback for the researcher. The result of the evaluation was analyzed in a statistical description which was presented in Table 4.8. From the result, the mean ranged from 3 to 3.7 on the scale of 4. It meant that the designed materials were considered appropriate and acceptable but still needed some revisions. The comments and feedback given by the evaluators were useful to revise and improve the materials. 74 Table 4.8 The Result of the Evaluation Questionnaires No Statements N Points of Agreement Central Tendency Mn 1 2 3 4 1. Topics The topics are well selected 3 1 2 3.7 The topics are well arranged 3 1 2 3.7 2. Learning Objectives Learning objectives are clear and precise 3 1 2 3.7 Learning objectives are varied 3 1 2 3.7 Learning objectives are well formulated 3 1 2 3.7 Learning objectives are well arranged 3 1 2 3.7 3. Reading Texts The reading texts are easy to understand 3 3 3 The reading texts are suitable for students 3 2 1 3.3 4. Writing Tasks The writing tasks are easy to understand 3 2 1 3.3 The writing tasks help the students to improve their writing competency 3 1 2 3.7 The writing tasks are suitable for History Department students 3 1 2 3.7 5. The Exercises The pre-activities are well arranged to stimulate students‟ prior knowledge 3 3 3 The exercises are helpful for students to understand the topic better 3 2 1 3.3 The exercises are well developed 3 1 2 3.7 6. The Instructions The instructions in each exercise and task are clear 3 2 1 3.3 The instructions are easy to understand 3 2 1 3.3 7. The Layout The pictures are interesting to attract the students‟ attention 3 1 2 3.7 The color are attractive enough for students 3 1 2 3.7 8. The Designed Materials The designed materials are interesting 3 1 2 3.7 The designed materials are relevant to history field of study 3 2 1 3.3 The designed materials are suitable for the first semester students of History Department 3 3 3 75 No Statements N Points of Agreement Central Tendency Mn 1 2 3 4 The designed materials help the students raise their reading and writing skillsabilities 3 2 1 3.3 The designed materials are well elaborated 3 2 1 3.3 The designed materials integrate the language learning and the content 3 1 2 3.7 There were two questions addressed to the evaluators in the open-ended questions. First were the strengths and the weaknesses of the designed materials and second were suggestions to improve the materials. The evaluators answered the questions and the feedback given was satisfying. They also gave some comments and suggestions to revise and improve the designed materials. The comments and suggestions given by the evaluators were listed as follow.  The strengths of the designed materials 1 The designed materials are suitable for the students and the learning activities are varied which are good for the students. 2 The topics are great value in accordance to the disciplinary orientation of students. 3 The materials are very interesting and attractive. The instructions are clear enough for the first semester students of the History Department. The pictures and the layout make the designed materials even better.  The weaknesses of the designed materials 1 Some contexts seemed to be taken from various sources but the sources were not written. 76 2 There were some grammatical mistakes and incorrect punctuations. 3 The topicsthemes need to be selected based on the students‟ interest and the level of difficulty needs to be adjusted from simple to complex.  Comments and suggestions 1 Make the texts more manageable for the first semester students of History Department. Divide the sentences in the reading texts to simpler ones for the students so that they can comprehend the texts better. 2 It is imperative to elaborate the pre-reading activities. To focus on the comprehension-based learning, the pre-reading activities will be much more relevant as far as they are integrated with the reading passages. 3 The exercises are helpful for the students because the designed materials were arranged from simple to complex. They are also comprehensive enough to practice reading and writing skills. It will be better to add some listening exercises so that the designed materials will be more complete and suitable for the students. All of the comments and suggestions from the evaluators were useful for the researcher to revise the designed materials. The feedback would be used as the considerations in revising and improving the designed materials.

5. Main Product Revision

a. Revising the Designed Materials Additional

The researcher revised the materials based on the feedback given by the evaluators. This revision was needed to make sure that the materials would be suitable for the first semester students of the History Department. First, the 77 researcher corrected the grammatical mistakes and incorrect punctuations. The researcher rechecked all of the sentences to avoid the same mistakes. Then, the researcher revised the instructions in some parts of designed materials so that the instructions would be clearer for the students. The researcher also edited some of the reading texts so that they would be more readable and manageable for the students. To make the designed materials more interesting, the researcher put some vocabulary games such as crossword and word search. Actually, one of the lecturer suggested that listening section should be given in the designed materials, however, the researcher did not take the suggestions since the materials were designed to help the students achieve the targets of English learning which were reading comprehension and writing competence. In conclusion, this step was done to present the final revision of the designed materials. It consisted of four units namely Kris, R. A. Kartini, Ullen Sentalu, and Borobudur Temple. Each unit consisted of seven stages of learning namely Let’s Ask, Let’s Guess, Let’s Read, Let’s Think, Let’s Learn, Let’s Write, and Let’s Act. Here is the description of the stages in the designed materials for the first semester students of the History Department.  Let‟s Ask This stage introduces the students about the topic that they are going to learn. Few questions are provided to give brainstorm and stimulate students‟ background information of the topic.