Guttman’s scalogram

2.1 Guttman’s scalogram

A basic idea of IRT models is that the answer of a student v to an item i reflects in some respect a relation between the student and the item. If the answer is correct, this is an indication that the student dominates the item. If the item is about mathematics, then a correct answer indicates that the student has more ability than required by the item; if the answer is incorrect, the item dominates the student, meaning that the student has less ability than required by the task. Using terms such as ‘more’ and ‘less’ implies an order relation, and a geometric representation of this order is the use of a directed line or dimension, where the terms of the comparison are represented as points. This idea is due to Guttman (1950) and is the basis of scalogram analysis. The idea is displayed graphically in Figure 8.1: the horizontal line represents the ability continuum,

158 Different methodological orientations

yzi

m Figure 8.1 Graphical representation of scalogram analysis

the vertical lines correspond to the position of the items i, j and m and the letters v, w, y and z represent the positions of four students. The relative positions of the letters to each other represent the dominance relation: the right-most position reflects the higher ability.

The theory of scalogram analysis amounts to the following: • If the positions of student and item points are known, then the behaviour

– the item answer – is known: if a student position dominates the item position, then the answer is correct; otherwise it is incorrect. This means that Guttman’s model is deterministic.

• Items and students are positioned on the same continuum. From the view- point of the student the position reflects his ability; the position of the item reflects its difficulty, but unlike in Classical Test Theory, the difficulty does not represent a proportion of correct answers in some population; it reflects the required ability to grant a correct response.

• The theory is testable. In Figure 8.1, there is no point on the continuum that dominates the position of item j and that at the same time is dominated by the position of item i. Or, in behavioural terms, it is not possible to give

a correct answer to a difficult item and an incorrect answer to an easier item. With a test of three items, it follows that there are only four response patterns possible: (0,0,0), (1,0,0), (1,1,0) and (1,1,1), represented in Figure

8.1 by the letters v, w, y and z respectively. In the general case with a test of k items, only k + 1 different response patterns are allowed, while the number of possible answers is 2 k . Guttman’s theory leaves no room for ‘errors’, and consequently, it has to be rejected in most of the cases.

A scalogram analysis amounts to finding the correct order of the items and students from a given data set. The result is an ordinal scale, and scale values given to students can only be used as ordinal numbers. Assigning the values

1, 2, 3 and 4 to the students v, w, y and z reflects their ordering in ability, but the same ordering is reflected by the scale values 1, 7, 32 and 107, and there is nothing in the theory that justifies the preference of one above the other.

The vulnerable aspect of Guttmans’s theory is its deterministic character. In probabilistic terms, it only uses probabilities of zero and one: if the item dominates the student, the probability of a correct answer is zero; otherwise it is one. Most IRT models in use can be seen as a relaxation of this deterministic

Using Item Response Theory 159 feature, as they describe in much detail the probability of a correct answer as a

function of the relative position of student and item points on the ability continuum. A simple model is described in the next section.