32
4. Evaluating the Design
In evaluating the design, the researcher distributed questionnaires as formative evaluation. The researcher evaluated the design with the help from
evaluators‟ group. They were two English teachers of SMP Pangudi Luhur 1 Yogyakarta and two English lecturers of English Language Education Study
Program of Sanata Dharma University. The evaluation had fifteen close-ended questions and five open-ended questions covering the evaluation stages. The list
of questions can be seen in appendix B.
B. Research Setting
First, the researcher started this research by gathering data interview in SMP Pangudi Luhur 1 Yogyakarta at Jalan Timoho II No. 29. The researcher
conducted the interview on April 2014. Second, the distribution of questionnaires for students
‟ group held on April-May 2014 in SMP Pangudi Luhur 1 Yogyakarta. And the last, the distribution of evaluation questionnaires for evaluators
‟ group conducted at different places and different time periods. In June 2014, the first
evaluation conducted at school and in July 2014, the second evaluation conducted at Sanata Dharma University.
C. Research Participants
This section elaborates on the participants of the research as well as in research and development methodology. The researcher divided the participants
into three groups. They were:
33
1. Teacher Group
The first group called teachers‟ group. Two English teachers participated
in this interview as interviewees. Both two English teachers taught in SMP Pangudi Luhur 1 Yogyakarta for seventh grade students. The researcher
interviewed them with eight open-ended questions. Indeed as data analyses, the researcher took data interview results from the interviewees.
2. Student Group
The second group called as students ‟ group. The seventh grade students of
SMP Pangudi Luhur 1 Yogyakarta became the participants in answering the questionnaires. They were from three different classes, e.g., 7 B, E, and G. Each
class consisted of approximately forty students and the total amount of students was one hundred and twenty seven students as the research participants.
Therefore, the researcher took data questionnaire results to find the need analysis.
3. Evaluator Group
The third group called as evaluators ‟ group. They were two English
teachers of SMP Pangudi Luhur 1 Yogyakarta and two lecturers of English Language Education Study Program of Sanata Dharma University. They
evaluated the students‟ exercises design. Finally, the researcher considered the
result of evaluation as data revision to improve the design as final product revision.
34
D. Instruments and Data Gathering Technique
In order to acquire the data, the researcher implemented survey technique in this research within three instruments. They were conducting interview,
distributing the questionnaires and evaluation questionnaires. The description and example of those three instruments discussed as follows:
1. Interview
In the interview process, the researcher asked in Indonesian language to get more detail information. The interview consisted of eight open-ended
questions covering the need analysis in this research. The example of interview‟s
questions as in appendix B the English version: 1
Does reading skill in curriculum 2013 become a dominant skill besides other skills, e.g. speaking, listening, and writing?
2 What are your ways in designing materials for reading which relevant
to the four main competencies of curriculum 2013? In April 2014, the researcher conducted the interview for two English
teachers of SMP Pangudi Luhur 1 Yogyakarta. The result of the interview analyzed in a form of narrative description.
2. Questionnaire
In questionnaire, the researcher applied Indonesian language as the questionsstatements. The purpose is that the students may get better
understanding in each statement. The example of intervie w‟s questions as in
appendix B:
35
Table 6. Questionnaire No.
Pernyataan Poin Persetujuan
1 2
4 5
1. Kegiatan membaca akan lebih menarik apabila
latihan soal yang diberikan berupa teks yang berkaitan dengan mata pelajaran lainnya, seperti
ilmu pengetahuan alam dan sosial, sejarah, dan lain sebagainya.
1 2
4 5
2. Saya sangat membutuhkan latihan kosa-kata
sebelum kegiatan membaca sehingga saya mampu memahami bacaan yang diberikan.
1 2
4 5
Ary, Jacobs, Sorenses 2010 define a Likert scale or summated rating scale uses
to assess students‟ attitudes related to the topic in the research which is going to be asked in the questionnaire. Again, Best 1970 states
that “the instrument yields a total score for each respondent, and a discussion of each
individual item, while possible, is not necessary” p. 175. In scoring the Likert scale, the researcher cited from
Ary et.al 2010, “strongly agree is scored 5, agree is scored 4, undecided is scored 3, disagree is scored 2, and strongly disagree is
scored 1” p. 210. The agreement points in this research can be seen in Table 7
below.
Table 7. The Meaning of Agreement Points Point’s
Agreement Response Category
Abbreviation
1 Strongly disagree
SD 2
Disagree D
4 Agree
A 5
Strongly agree SA
Therefore, the purpose of agreement points is to arrange the students who chose in agree part and disagree part.
36
3. Evaluation
The researcher conducted the evaluation which was divided into three parts. The first part had ten close-ended-statements about the look of reading
exercises. The second part had five close-ended statements about the Hot-Potatoes program works in the website. Both parts adopted Likert scale in measuring the
evaluation and the agreement points as the same as in Table 7 above. The last part was the evaluator
‟s feedback which had five open-ended questions. The evaluators wrote feedback, opinion, and suggestion as the design improvement.
The meaning of agreement points can be seen in Table 8 below.
Table 8. The Meaning of Agreement Points Point’s
Agreement Response Category
Abbreviation
1 Strongly disagree
SD 2
Disagree D
3 Agree
A 4
Strongly agree SA
Therefore, the purpose of agreement points was to arrange the students who chose in agree part and disagree part.
E. Data Analysis Technique
The researcher divided this part into three. They were data interview, questionnaire and evaluation.
1. Interview
In analysing the data interview, the researcher translated and summarized the data in narrative description using English. The response of interviewees can
be seen in appendix C. Firstly, the researcher transcribed the results of the data