The checklist was used in analyzing the textbooks. The checklist consisted of three aspects with seven sub aspects for textbook evaluation.
Those aspects then were broken down into some sub aspects. The sub aspects were divided into some areas. Finally, the areas were elaborated into some
items which became the criteria for evaluating the English textbooks. The items were elaborated further into some belief statements to make it easier for
evaluation. 6. Interpreting the findings and stating conclusions
The interpretations and explanations were in narrative form Ary. et. al, 2010: 32. The criterion which was met by the textbook was given a tick
and was valued 1 point, while a criterion which was not met was given a cross and was valued 0 point. In the checklist, only the column was ticked or
crossed. The criteria which were met and the ones which were not were calculated in the form of percentage. The percentage was gained from the total
of criteria points which were not met in a textbook for each sub aspect divided by the total of criteria points in each sub aspect. Further descriptions were
done afterwards. . Writing down the report
After doing some procedural steps above in an orderly way, the last step namely writing down the report was conducted. This report explained
each part and procedure of the study and the findings in detail.
D. Research Instrument
The instrument used in this research is human instrument. Therefore, the researcher was the primary instrument Janesick , 1994 in Ary et. al.,
2002: 426. Since understanding was the goal of this research, the human instrument, which was able to be immediately responsive and adaptive, was
the ideal means of collecting and analyzing data Merriam, 2002: 5. The two textbooks used in most of Senior High Schools in Yogyakarta
were analyzed. They were written for the tenth graders and based on two different curricula, namely KTSP and 2013 curriculum, which were based on
genre-based approach. Concerning that content analysis focuses on analyzing and interpreting recorded material which might be in the form of public
records, textbooks, letters, films, tapes, diaries, themes, reports, or other documents Ary. Et. al., 2010: 29, this research analyzed the textbooks in
terms of content, language, and presentation. Apart from the researcher as the primary instrument, the textbooks as
the documents to analyze were the second instrument. The documents used in this research were a primary source, for the researcher analyzed the textbooks
directly. In this research, the researcher-generated documents for the subject by
providing tables of checklist to observe and evaluate the book in some areas were prepared Aly et. al, 2010: 442. The areas and belief statements in the
checklist were based on the textbook evaluation criteria issued by Pusbuk in 200. It was actually still based on KTSP Curriculum. Therefore some
modification to the aspects of content, language, and presentation was made to
suit 2013 Curriculum. The modification in presentation aspect was done by adding some criteria which represent scientific approach as the core of 2013
Curriculum. The lacks in language and content aspect were overcome by adding some criteria from the EFL experts such as Ur 2009, Cunningsworth
in Richards 2001 and Nation and Macalister 2010. There were three aspects of the textbooks to be evaluated in this
research namely content aspect, language aspect, and presentation aspect. The complete checklists for evaluating the textbooks were attached in the
appendices.
1. Content Aspect
The content aspect had three sub aspects namely the relevance of materials to the curriculum, material accuracy, and supporting learning
materials. The sub aspect of relevance of materials to the curriculum had three
areas with twelve items. The areas to be evaluated in these sub aspects were material completeness, material depth, and material organization. The items
for material completeness and material depth were taken from the textbook evaluation criteria formulated by Pusbuk 200; the items for material
organization were adapted from the ones formulated by Nation and Macalister 2010 and Penny Ur 2009. The material completeness was also modified
by adding criteria on presentational texts and character building as the core of 2013 Curriculum.
