33
purposes of using a test as research instrument, especially to answer the second research question, the researcher decided to use five types of test item, namely
transformation test, completion test, word order test, multiple choice test, and translation test.
In order to collect as much data as possible and to fulfill the characteristics of errors in constructing interrogative word question, the researcher preferred to
construct the test items with equal number of each type. The researcher made 50 test items which were divided equally to five types of test item. Thus, each type of
test item contained 10 items. To make it clearer, the researcher provided the table of test items distribution as follows:
Table 3.1 Distribution of Test Items
Transformation Completion
Word Order
Multiple Choice
Translation Test
10 10
10 10
10
3. Validity and Reliability of the Test
When designing a test as the research instrument the researcher should consider two essential requirements of a test. They are test validity and reliability.
If the test used in the research is valid and reliable, the data obtained from the test is considered valid and reliable to answer the research questions.
a. Validity
“Validity is an important key of an effective research...However, it is impossible for research to be
100 valid” Cohen, Manion, Morrison, 2000, p. 105. Besides, Ary, Jacobs, and Razavich argue that validity is always specific. It
34
is specific to the particular purpose of the instrument that is being used. Nevertheless, the instrument validity may not be valid in different situation for
different purpose 1990. Hatch and Farhady add that validity of a test refers to the extent to which the results of the procedure serve the uses of what they were
intended. It refers to the results not to the test 1982.
1 Content Validity
Based on the theory proposed by McNamara, content validity refers to the relevance of the test content to what is measured 2008. It means that the test
given to the samples of research should be relevant to what the researcher wants to measure. Therefore, the test items should be designed according to the purpose
why the test is given. In addition, Hatch and Farhady 1982 state the focus of the content validity is on the adequacy of the sample rather than the appearance of the
test. Considering a test should contain content validity, the test items given to
the samples of research, which were students of PMat 2010, were designed to represent the content validity of a language test. Besides, the test items were also
constructed based on the purposes why the test was given. In this research, the purposes of the test were to discover the errors the samples made regarding the
construction of interrogative word questions and the sources of the errors. Furthermore, the test items used in this research contained the material, which
would be measured, related to the use of
what, who, where, when, why,
and
how
. As a result, the test items were valid enough to measure what would be measured.
35
2 Construct Validity
Construct validity of a test is defined as the criterion which is used to validate large scale standardized test of proficiency Brown, 2004. Further,
construct validity is used to determine how well the test scores represent the learning objectives and how well the test scores predict the particular performance
Hatch Farhady, 1982. In addition, McNamara proposes a theory that test construct underlies the ability which is being measured by the test 2008.
In order to represent the construct validity of a good test, the test items were designed to demonstrate what would be measured. In this case, the purposes
of the test were to define the errors and the sources of errors in English interrogative word questions made by students of PMat 2010. Thus, the test items
were divided into five types namely transformation, completion, word-order, multiple-choice, and translation.
The first part of the test, which is named transformation test, requires the samples of research to transform the statements to appropriate interrogative word
questions which are used to question the subject, object, adverb, or quantifier. In the second part, which is called completion test, the samples of research are given
order to complete the 10 items of incomplete interrogative word question. What they should write to complete the interrogative word questions are only the
interrogative words. Further, the third part represents the word-order test. In this part, the
samples of research are required to construct correct interrogative word questions
36
according to the jumbled words given in each number. The fourth part, which is named multiple-choice test, requires the samples of research to pick one best
answer dealing with the use of interrogative words. At last, the translation test is constructed to know how far the samples of
research understand about the construction of interrogative word questions in both languages they use, which are English and Bahasa Indonesia. In this part, students
of PMat 2010 are required to translate 5 items of English interrogative word question to Bahasa Indonesia and 5 items of interrogative word question in
Bahasa Indonesia to English.
3 Face Validity
Mousavi 2002 as cited in Brown 2004 states that face validity refers to the degree to which the test appears to measure the proficiency or ability based on
the examinees who take it. It means that the face validity of the test represent the relevance of the test in the eye of the examinees or test takers. As a matter of fact,
validity of a test is not something that can be tested scientifically, yet it can be measured if the learners or test takers encounter the uncomplicated items, clear
directions, and suitable difficulty level. Considering that matter, the researcher then constructed 50 test items
which were uncomplicated and suitable to the level of English proficiency of the samples. The test items were constructed based on the preliminary observation
done before the test was given. Moreover, the whole test items involve daily used words in which the samples have already mastered. Some test items were also
37
adapted and taken from the module used in the English course, in which students of PMat 2010 were taking at that time. Besides, to support high face validity, the
researcher also consulted the test items to the advisor and revised them two times.
b. Reliability
