28 recognize it as familiar type of task, and engage the relevant cognitive and language
processing skills – from their training – to attempt the item.
2.10.2. Guiding Language Versus samples
There are many styles and layouts for test specs. All test specifications have two components: samples of the items or tasks we intend to produce and guiding language about the
samples. Guiding language comprises all parts of the test spec other than the sample itself. For the above seat-belt sample, guiding language might include some of these key points:
[1] This is a four-option multiple-choice test question. [2] The stem shall be a statement followed by a question about the statement.
[3] Each choice shall be plausible against real-world knowledge, and each choice shall be internally grammatical.
[4] The key shall be the only inference that is feasible from the statement in the stem. [5] Each distracter shall be a slight variation from the feasible inference from the stem;
that is to say, close reading of all tour choices is necessary in order to get the correct answer.
Taking a cue from Ruch, we can assume one more important bit of highly contextualized guiding language:
[6a] It is assumed that test takers are intimately familiar with this item type, so that they see instantly what kind of task they are being asked to perform; that is to say, the method
of the item is transparent to the skills it seeks to measure. An alternative would be to assume that our test takers are not familiar with this item type:
www.eprints.undip.ac.id © Master Program in Linguistics, Diponegoro University
29 [6b] Test takers may or may not be familiar with this item type. The level of familiarity is
not of any importance. The focus of this item is close reading, because the item must function as an assessment of proficiency, preferably of high-level proficiency.
2.10.3. Congruence Fit-To-Spec
The seat-belt item was first presented, above, without any guiding language. We then encouraged a critical reflection about the item. This is rather typical in test settings. We have
test tasks in front of us, and somehow we are not happy with what we see. Our next step above was to induce some guiding language such that items equivalent
to the seat-belt question could be written. For example, something like this should be equivalent
– it appears to follow our evolving spec here:
The vast majority of parents in a recent survey favourstricter attendance regulations at their childrens schools. Which of the following could be true?
a Most parents want stricter attendance rules. b Many parents want stricter attendance rules.
c Only a few parents think current attendance rules are acceptable. d Some parents think current attendance rules are acceptable.
2.10.4. How Do Test Questions Originate?
