E. Data Analysis Techniques
The analysis of this research focused on the document of national final examination year 2007 for vocational high schools. Each test item was observed
in reference to the validity of language test, so as to find out its consistency and relevance to the test specifications. In other words, this research attempted to seek
if the national final examination has really measured what it purported to measure. Best 1977: 129 says that researches generally
“classify and evaluate the contents of documents according to established criteria”. The established criteria
in this research were those of language test validity. The validity of the national final examination was evaluated in accordance with the content validity and face
validity. These were the established criteria that the researcher used for analyzing the national final examination:
1. Face Validity Alderson et al. 1995: 172 highlight th
at, as “tests „surface credibility or public acceptability‟”, face validity may take into account some „lay‟ people‟s
judgments against the test. Therefore, the researcher obtained descriptive data by asking
for some people‟s opinion about the face validity of the national final examination year 2007 for vocational high schools beside his own opinion. Some
lecturers and students of English education study program were asked to help him with this part of validity investigation. The researcher conducted short interviews
to them asking whether the document of the test was, in their opinion, face valid. The researcher intended to gather their opinion on the face validity after they had
been presented with the copy of the national final examination. They did not need
to look carefully at the content or construct of the test items. Their opinion was just at the “surface-level” aspect of the test.
2. Content Validity Related test specifications are necessary in measuring
a test‟s content validity. Hughes 1989: 23 suggests researchers to prepare the test specifications
before conducting the analysis of the content validity. Since this research investigated the validity of national final examination year 2007 for vocational
high schools, the test tasks were derived from Competency-based Curriculum and relevant syllabus, which turned out to agree with
Standard of Graduate‟s Competencies Standar Kompetensi Lulusan year 20062007.
The test specifications of national final examination year 2007 for reading section are shown in Table 3.1 below.
Table 3.1 Test Specifications of Reading Section S
TANDARD OF
G
RADUATE
’
S
C
OMPETENCIES
T
OPICS
2. Students are able to comprehend the
meaning of interpersonal and
transactional written discourse, whether
formal or informal, that is related to
daily life, occupation, and
profession. a. Finding main idea, implicitexplicit detailed information,
derivation, synonym, antonym, word meaning from particular written texts.
b. Finding wordsphrases within a written text or short conversation inappropriate to grammar or its context.
c. Determining appropriate response to particular expressions:
1. greetings 2. introduction
3. describing things, people, situation, process 4. comparing two or more things, people, or situation
5. handling guestscustomers directly and on the telephone 6. imperative: order, request, prohibition, direction
7. offering
S
TANDARD OF
G
RADUATE
’
S
C
OMPETENCIES
T
OPICS
8. showing agreementdisagreement 9. making reservation
10. asking for and giving suggestion 11. asking for and giving opinioninformation
12. asking for and giving advice 13. showing ability
14. asking for and giving permission 15. showing possibility
16. showing obligation 17. dailyprogressivepastfuturepresent perfect progressive
activitiesthe ones that are repeated in the past. 18. thanking
19. making invitation 20. congratulating
The standard of graduate‟s competencies of reading skill is elaborated in
some more specific topics as shown in the table above. The standard signifies that students are supposed to be able to comprehend the meaning of interpersonal and
transactional written context of discourse, whether it is formal or informal, which is related to daily life, occupation, and profession. Transactional meaning is a
variation of interpersonal meaning. Interpersonal meaning of a clause is obvious from its mood, which consists of a subject and finite verb. A mood can make a
distinction on one‟s intention or mood, whether it is informing the listener, asking, or giving an order.
The test specifications of the reading section above thus serve as the “standard of comparison” for the researcher to determine the content
representativeness of the test. The test content representativeness means the adequacy of content sampling within the test. In other words, the reading section
of the national final examination year 2007 for vocational high school must contain as sufficient content samples as possible of the topics in the test
specifications above in order to fulfill its content validity. During the content analysis, the researcher follows the division of the
reading section of the test and compares each of its part with three main topics of learning as shown in Table 3.1. The reading section is divided into reading
comprehension, error identification, and incomplete dialogues. Topics of learning that must be covered in the reading comprehension test items 36-50 are finding
main idea, implicitexplicit detailed information, derivation, synonym, antonym, or word meaning from particular written texts. Error identification part covers the
learning topic of finding wordsphrases within a written text or short conversation that are inappropriate to grammar or its context. The incomplete dialogues part is
related to the learning topic of determining appropriate response to particular expressions. The latter contains some more specific topics of learning. Those
more specific topics of learning are listed as sub topics. They are numbered as 1 up to 20 in Table 3.1.
The researcher‟s task was then to compare each test item in the reading section with the test specifications. This was how he investigated
the test‟s content relevance. However, not everything that appeared in the test specifications
was included in the test. What the researcher attempted to find out was the extent to which they appear in the test. In this part of the research, the researcher
attempted to find out whether the learning topics shown in Table 3.1 are covered
in each part of the reading sections. The researcher also elicits any test item that deviates from the intended topic of learning to be claimed as invalid.
As explained previously, the analysis was done by seeking test items relevance to the teaching materials as summarized in the test specifications. The
researcher worked on the test paper, analyzing each test item to find out if it really represent and is relevant to the test specifications. Data that were gathered in this
type of analysis were thus descriptive because the researcher dealt with some notes and explanations that he has made during the analysis. The data was then
arranged into a more systematic description in order to become the answer to question of research problem.
F. Research Procedures
Kategori