38
the researcher calculated the results. The final results were presented into percentage. The results were differentiated into two, first was the result of
checking content validity of both reading and listening items based on Competence Standard and Basic Competence then the second was the result of
checking content validity of both listening and reading items based on Graduate Competence Standard. Since there were five reading test versions of the National
Examination utilized in this research, there were five kinds of results which were presented into percentage for the reading section.
3. Checklists to check authenticity of English Items on listening and reading
section of National Examination
The authenticity indicators utilized in this research were developed from Brown’s theory 2004 about authentic language test. The researcher
differentiated the checklists used to assess the listening and reading test items. Since the listening items were provided in only one test version and the items
were integrated with the passages, the researcher integrated the assessment for testing the test tasks and the test texts in one kind of checklist while the reading
items were assessed in two kinds of checklists; the checklist to check authenticity of the test tasks of the reading items and the checklist to check authenticity of the
test texts of the reading items. For checking the listening section, each test item was checked with the
five authenticity indicators stated by Brown 2004, namely the natural language use, contextualized items, the relevance between the test topics and the learners,
the representativeness of thematic items organization, and real-world
39
representativeness. Since the representativeness of thematic items organization should not be presented in each item, it was not an obligatory to fulfill this
indicator. If the set of listening items had representativeness for this indicator, therefore the set of listening items was considered fulfilling this indicator.
Table 3.2 The Sample of Authenticity Checklist
Test item number The Criteria of Authenticity
Natural language use Contextualized Items
The relevant topic Episodic item organization
Real-world representativeness Natural language use
Contextualized Items The relevant topic
Episodic item organization Real-world representativeness
Natural language use Contextualized Items
The relevant topic Episodic item organization
Real-world representativeness
This kind of checklists was filled by giving tick on the provided
column. The presence of the tick meant that the listening test materials of National Examination reflected the indicators of authenticity while the absence
of the tick on the provided criteria meant that the listening test materials of National Examination did not reflect the indicators of authenticity. After
fulfilling the checklists and getting the data, the researcher calculated the results. The final results were presented into percentage.
In order to check the reading section, the test tasks of the reading test items were analyzed based on the five authenticity indicators which were stated by
Brown 2004. However, the researcher firstly selected the reading passages of
40
each reading test versions since the researcher found some similar passages in some test versions. After selecting the reading passages, the researcher finally
found 50 different kinds of reading passages with 123 test instructions. The researcher employed two kinds of checklists in order to analyze the authenticity of
English reading test tasks of National Examination. They are checklists to check the reading tasks and to check the reading texts and they were taken from
Bachman and Palmer’s theory. First checklist was employed to check the reading test tasks. It referred to
the reading test instructions and the optional answers. The sample of the checklist was almost similar to the checklist of the listening section see appendix four on
pages 122 up to 137. The authenticity indicators utilized in the checklists were: the natural language use, contextualized items, the relevance between the test
topics and the learners, the representativeness of thematic items organization, and real-world representativeness. Since the representativeness of thematic items
organization should not be presented in each item, it was not an obligatory to fulfill this indicator. If the set of reading test tasks had representativeness for this
indicator, therefore the set of reading test tasks was considered fulfilling this indicator.
The researcher filled the checklist by giving tick on the provided column. The presence of the tick meant that the reading test tasks of National
Examination reflected the indicators of authenticity. Meanwhile, the absence of the tick on the provided criteria meant that the reading test tasks of National
Examination did not reflect the indicators of authenticity. After fulfilling the
41
checklists and getting the data, the researcher calculated the results and the final results were then presented into percentage.
Afterwards, the second checklist was utilized to check the test texts of the reading section. The passages which were analyzed are the 50 passages with 132
test instructions. The sample of the checklist employed to check the reading texts on the English test items could be seen in the table 3.3 or see in appendix four on
pages 138 up to 141.
Table 3.3 The Sample of Authenticity of the Test Text Checklist
Test passages The Criteria of Authenticity
Natural language use The relevant topic
Real-world representativeness Natural language use
The relevant topic Real-world representativeness
Natural language use The relevant topic
Real-world representativeness
From the table 3.3, it is seen that the researcher modified the authenticity
indicators stated by Brown 2004. In this checklist, the researcher employed three of the five authenticity indicators, namely the natural language use, the relevance
between the topics and the learners, and the real-world representativeness. The researcher filled the checklist by giving tick on the provided column. The
presence of the tick meant that the reading test passages of National Examination reflect the indicators of authenticity while the absence of the tick
on the provided criteria meant that the reading test passages of National Examination did not reflect the indicators of authenticity. After fulfilling the
checklists and getting the data, the researcher calculated the results. The final
42
results were presented into percentage.
F. Research Procedures
