20 for the respondents to explain responses given in written or other formats. In
validity perspective, it is advantageous because reasoning and explanation skills are in the domain.
e. Portfolio-based Assessments
Portfolio-based assessment provides a comprehensive picture of proficiencies the students made. It is often promoted as the best means for
comprehensive documentation of evolving skills and knowledge in a particular area. According to Arter and Spandel 1992, p. 36,
a portfolio is a purposeful collection of student work that tells the story of a student’s efforts, progress, or achievement in a given areas. This
collection must include student participation in selection of portfolio contents; the guidelines for selection; the criteria for judging merit; and
evidence of student self-reflection.
3. Principles of Language Assessment
Brown 2004, p. 19 stated five principles of language assessment namely practicality, reliability, validity, authenticity, and washback. Since this study
places its attention specifically on validity of assessment, the writer will explain the validity in bigger portion than other principles. The principles explained
bellow are mainly taken from Brown’s book, Language Assessment: Principles and Classroom Practices 2004, p. 19-30.
a. Practicality
The first principle is practicality. An effective test is practical. This means that it is not excessively expensive, it stays within appropriate time
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
21 constraints, it is relatively easy to administer, and it has scoringevaluation
procedure that is specific and time-efficient.
b. Reliability
A reliable test is consistent and dependable. It means that the same test should yield similar result when it will be given to the same students or matched
students on two different occasions.
c. Validity
The next principle is validity. This principle becomes the basis of this study. Therefore, as stated above, the writer will give bigger portion to explain
about this assessment principle. The most important quality to consider when producing or selecting an
assessment procedure is validity, which refers to the meaningfulness and appropriateness of the uses and interpretations to be made of assessment result
Miller et al., 2009. Bachman 1990 also stated that “validity has been identified as the most important quality of test use, which concerns the extent to which
meaningful inferences can be drawn from test scores” as cited in Liao, 2004, p. 1. According to Gronlund 1998 validity is the most complex criterion of an
effective test and arguably the most important principle as cited in Brown, 2004, p. 22. Validity is defined as “the extent to which inferences made from
assessment results are appropriate, meaningful, and useful in terms of the purpose of the assessment” Brown, 2004, p. 22.
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
22 Brown 2006 stated that validity is important especially when it is
involved in the decisions that teachers regularly make about their students. Further he stated that teachers want to make the decisions of their admission, placement,
achievement, and diagnostic based on the test or assessment which actually testing what they claim to measure. Therefore, teachers should consider their assessment
validity. The assessment will be considered valid if it measures the objective of the course Siddiek, 2010 because “aimless is the most single cause of
ineffectiveness in teaching and of frustration of education efforts” Cliff, 1981, p. 27, as cited by Siddiek, 2010, p. 135.
“Validity is the quality of the interpretation of the result rather than of the assessment itself, that its presence is a matter of degree, that it is always specific
to some particular interpretation or use, and that it is a unitary concept” Miller et al., 2009, p. 102. What validated is not merely the assessment procedures but the
appropriateness of the interpretation and use of the result of an assessment procedure. Validity is best considered in terms of categories that specify the
degree, namely high validity, moderate validity, and low validity Miller et al., 2009. It is better to avoid thinking of assessment result as valid or invalid.
There is no final, absolute measure of validity, but several different kinds of evidence may be evoked in support. They are content-related evidence,
criterion-related evidence, construct-related evidence, consequential validity, and face validity. The research focuses on content validity of an assessment, so the
explanation about content validity will be in bigger portion than other validities.
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
23
1 Content-Related Evidence
Content is simply defined as something what to teach or what the students need to learn. Price and Nelson 2011 stated that planning for instruction
begins with thinking about content. Before making decision about how to teach, teachers should decide what to teach or what students need to learn p. 3.
Furthermore, Price et al. 2011 stated that it is impossible to achieve the intended result from teaching when the content, what to teach or what the students need to
learn, is not exactly stated or determined. The content of language learning cannot be separated from the four basic language skills, namely reading, writing,
speaking, and listening. Those four skills are integrated in language learning Brown, 2004.
Musafi 2002 and Hughes 2003 stated that content-related evidence or content validity means that the test samples the subject matter about which
conclusions are to be drawn and it requires the test-taker to perform the behavior that is being measured as cited in Brown, 2004, p. 22. It means that if the test
can clearly define the achievement being measured, it fulfills the content-related evidence or content validity.
In this process, content validation becomes the main point. The goal, as stated by Brown 2006, will always be to establish an argument that the
assessment provides a representative sample of the content the test claims to measure. Further, the first step in assuring that the test or assessment is valid in its
content is by deciding what assessment should be designed to measure. Brown 2006 stated again that thinking about validity may initially involve in defining
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
24 what it is that the testers wanted to measure in the first place. The important point
in content validation is “to determine the extent to which a set of assessment tasks provides a relevant and representative sample of the domain of tasks about which
interpretations of assessment results are made” Miller et al., 2009, p. 75. Since language learning also involves the four basic language skills, the microskills and
macroskills of each skill are also involved in the analysis the list of microskills and macroskills of each content can be found in Appendix D.
2 Criterion-Related Evidence
Criterion-related evidence is the extent to which the “criterion” of the test has actually been reached. It is best demonstrated through a comparison of results
of an assessment with results of some other measure of the same criterion. For example, in a course unit whose objective is for students to be able to orally
produce voiced and voiceless stops in all possible phonetic environments, the result of one teacher’s unit test might be compared with an independent
assessment – possibly a commercially produced test in a textbook – of the same phonemic proficiency.
3 Construct-Related Evidence
The next principle of an assessment is construct-related evidence or constructs validity. A construct is any theory, hypothesis, or model that attempts
to explain observed phenomena in our universe of perceptions. “Proficiency” and “communicative competence” are linguistic constructs while “self-esteem” and
PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI PLAGIAT MERUPAKAN TINDAKAN TIDAK TERPUJI
25 “motivation” are psychological constructs. In the field of assessment construct
validity asks, “Does this test actually tap into the theoretical construct as it has been defined? Brown, 2004, p. 25”
4 Consequential Validity
Consequential validity encompasses all the consequences of a test, including such considerations as its accuracy in measuring intended criteria, its
impact on the preparation of test-takers, its effect on the learner, and the intended and unintended social consequences of a test’s interpretation and use Brown,
2004, p. 26.
5 Face Validity
Gronlund 1998 stated that face validity is the extent to which students view the assessment as fair, relevant, and useful for improving learning as cited
by Brown, 2004, p. 26. Brown also cited Mousavi’s 2002 definition about face validity.
“Face validity refers to the degree to which a test looks right, and appears to measure the knowledge of abilities it claims to measure, based on the
subjective judgment of the examinees who take it, the administrative personnel who decide on its use, and other psychometrically
unsophisticated observers” p. 244.
d. Authenticity
