Ability to RESULT OF EVALUATION ON THE PRELIMINARY PRODUCT

123 Indicators of language competence

7. Ability to

write vocabulary words in the right spelling Test items coverage Students’ response Experts’ judgment No. 24 successful accepted No. 25 successful accepted No. 26 successful accepted No. 27 successful accepted No. 28 successful accepted From the above table, it can be seen that all items have successfully fulfilled the expected results; all items are in the conformity of both the students’ response and the experts’ judgments. Therefore no revisions needed for the manifestation of this indicator. 2 Indicator 8: ability to build cohesion across sentences and parts of sentences with appropriate structure. Seven items are designed to manifest the indicator. These seven items are formatted in essay and divided into two parts. The first part is arranging words into sentences and the second part is arranging sentences into a passage. The result of the trial test is presented below: Table. 4.10 Conformity between students’ response and experts’ judgment for manifestation of indicator 7, item no 24-28 Table. . 4.11. Students’ response on. Text Production, writing part 2, indicator 8, item no 29-34 Indicator Writing , Part 2 Test itemsStudents 1 2 3 4 5 6 7 8 9 10 11 12 Qualification high high high high average average average average low low low low No. 29 1 1 1 1 1 1 6 No. 30 1 1 1 1 1 5 No. 31 1 1 No. 32 No. 33 1 1 2 Writing , Part 3 No. 34 15 15 3 15 1 1 1 15 1 67 TEXT PRODUCTION 8. Ability to build cohesion across sentences and parts of sentences with appropriate structure eg: appropriate punctuations, 124 It can be observed from the table above that the result is not quite satisfying. All items have not been successfully responded by the students with item no 32 as the worst result. A vivid description of the test item achievement can be seen from the performance of the trend line which is presented below: Figure 4.31. Trend line of students’ response for. Writing, item no 32 Figure 4.32. Trend line of students’ response for.Writing, item no 33 Figure 4.28. Trend line of students’ response for. Writing, item no 29 0.5 1 1.5 2 2.5 Low Middle High Correct response Figure 4.29. Trend line of students’ response for. Writing, item no 30 0.2 0.4 0.6 0.8 1 1.2 Low Middle High Correct response 0.2 0.4 0.6 0.8 1 Low Middle High Correct response Figure 4.30. Trend line of students’ response for. Writing, item no 31 Trend line Trend line Trend line Trend line 0.5 1 1.5 2 2.5 Low Middle High Correct response 10 20 30 40 50 60 Low Middle High Correct response Figure 4.33. Trend line of students’ response for.Writing, item no 34 Trend line Trend line 125 From the above graphs, it can be seen that the trend line performance of the test items showing normal patterns are for two items, namely item no 30 and 34. Here, the trend lines going from the low to high in rising slope show that the test item has been able to discriminate the criterion group. Yet the other four items, namely item no 29, 31, 32, and 33 have performed unexpected results with item no 32 gets the worst. They cannot discriminate between the criterion students and moreover, the task also cannot be solved by all the students zero result. Yet, to formulate the revision of the test items, the researcher needs to see the conformity of these results with the experts’ judgment which is presented below: Indicators of language competence 8. Build cohesion across sentences and parts of sentences with appropriate structure. Test items coverage Students’ response Experts’ judgment No. 29 Not successful Lacks of cohesions with the next sentences. No. 30 Successful Lacks of cohesions with the previous and next sentences. No. 31 Not successful Lacks of cohesions with the previous and next sentences. No. 32 Not successful Lacks of cohesions with the previous and next sentences. No. 33 Not successful Lacks of cohesions with the previous and next sentences. No. 34 Successful Lacks of cohesions with the previous sentences. It can be seen from the above table that most of the items are in conformity. Only two of the six items which have no conformity between the students’ responses with the experts’ judgments. Therefore, the improvement feedbacks given by the expert respondents will be analyzed to make decision on the item Table. 4.12. Conformity between students’ response and experts’ judgment for manifestation of indicator 8, item no 29-34 126 revision that will be elaborated more in the discussion of the preliminary product revision. 3 Indicator 9: ability to write their idea into sentences with appropriate meanings. Five items were provided to manifest the indicator. Here, the students were asked to write answers of the questions provided in sentences. The result of the trial test is presented below: It can be seen from the above table that the result is not too bad. Within the expected maximum score for each item is 36, the highest score is 20 and the lowest is 12. The result, indeed, does not perfectly fulfill the expected result. To see the achievement of the items more clearly, it is necessary to see the trend lines of the achievements which are presented below: Table. . 4.13. Students’ response on. Text production, writing part 4, indicator 9, item no 35-39 Indicator Writing , Part 4 Test itemsStudents 1 2 3 4 5 6 7 8 9 10 11 12 Qualification high high high high average average average average low low low low No.35 3 1 1 2 2 2 1 12 No. 36 3 1 2 3 3 1 2 1 1 17 No. 37 1 1 3 2 1 2 3 13 No. 38 3 3 3 3 2 3 3 20 No. 39 3 2 3 2 2 2 3 2 19 TEXT PRODUCTION 9. Ability to write their idea into sentences with appropriate meanings Figure 4.34. Trend line of students’ response for. Writing, item no 35 Figure 4.35. Trend line of students’ response for. Writing, item no 36 Trend line Trend line 127 It can be observed from the above graphs that the responses of the students during the trial test are quite good. The trend lines of the five items have performed rising slope which means the items has fulfilled the expected results. Yet, to decide whether revision is needed, the researcher will observe the conformity of the students’ response with the experts’ judgment. The data is presented below: Indicators of language competence 9. write their idea into sentences with appropriate meanings Test items coverage Students’ response Experts’ judgment No. 35 Successful The instruction should be clearer No, 36 Successful The instruction should be clearer No. 37 Successful The instruction should be clearer Figure 4.38. Trend line of students’ response for.Writing, item no 39 Table. 4.14. Conformity between students’ response and experts’ judgment for manifestation of indicator 9, item 35-39 Figure 4.36. Trend line of students’ response for.Writing, item no 37 Figure 4.37. Trend line of students’ response for.Writing, item no 38 Trend line Trend line Trend line 128 No. 38 Successful The instruction should be clearer No. 39 Successful The instruction should be clearer It is shown in table that there is no conformity between those two aspects. While all items are correctly responded by the students, on the contrary they are suggested for revision by the experts. Therefore, the decision of revision will be made by also digging out the improvement feedbacks given by the experts that will be elaborated in the discussion of the preliminary product revision.

3. Overview of the Design of the Testing Model.