Parameter estimation in fixed-effects models

2.1 Parameter estimation in fixed-effects models

In fixed-effects models, the item parameters as well as the value of the latent ability for each student v in the calibration sample are considered as unknown parameters that have to be estimated from the observed item responses in the calibration sample. Suppose one applies an incomplete design, where each student

IRT models 189 from a sample of, in total, n students answers a subset of, in total, k items. The

measurement model is the Rasch model. In a fixed-effects model the latent ability of each student is treated as a model parameter that has to be estimated from the data; so there are n + k – 1 free parameters, because one parameter can be freely fixed for normalization purposes.

Maximum likelihood estimates are those values of the model parameters that jointly make the likelihood function attain its maximal value. However, these values are the same as those that make the log-likelihood function maximal, because the logarithm is an increasing function of its argument. To find this maximum, the partial derivatives of the log-likelihood function are equated to zero. These equations are called the ‘likelihood equations’, and their solution yields the maximum likelihood estimates. The log-likelihood function in the Rasch model when applied to incomplete designs is given in equation (1), which is given here explicitly as a function of the unknown parameters.

ln L (,, β 2 … βθ k ,,, 1 … θ n ;,) xd

nk

s vv θ + ∑ t i ( − β i ) −

d vi 1 n 1 + exp( θ v − β i ). (1)

In the right-hand side of (1), the variable d vi is an indicator variable for the design, taking the value one if item i has been administered to student v, and zero otherwise. The symbol d in the left-hand side represents the matrix of indicator variables. The sufficient statistic s v and t i also have a slightly different meaning in incomplete designs:

s v = ∑ dx vi vi and t i = dx vi vi .

This implies that the test score of a student is the number of correctly answered items that have been administered. If d vi = 0, then x vi can have an arbitrary numeric value; it will never influence the outcome of an analysis since it is always

multiplied by zero. Notice that the parameter ␤ 1 does not appear in the left- hand side of (1), because it is fixed at some constant to normalize the solution – but it does appear in the right-hand side.

The derivation of the likelihood equations is beyond the scope of this chapter. Details can be found in Fischer (1974) and Molenaar (1995). It is, however, important to note that there are no explicit solutions for these equations but that they have to be approximated iteratively. This is true for the Rasch model, the simplest of all IRT models, and it holds a fortiori for all IRT models.

Solutions do not exist for ␪ v if student v has administered all items correctly or all incorrectly. Similarly, there is not a solution for ␤ i if the answers given to item i are all correct or all incorrect.

As the preceding restriction might be seen as a disadvantage of the Rasch model, it can be repaired relatively easily by increasing the sample size in such

190 Different methodological orientations

a way that at least one correct and one incorrect answer is given to each item. However, students with all items correct (getting a perfect score) or all incorrect (getting a zero score) have to be left out of the analysis.

The bad news about this fixed-effects Rasch model is that the parameter estimators are not consistent. If the sample size keeps growing, the item parameter estimates do not converge to their true value. Loosely speaking, the reason is that to collect more information about the item parameters, the sample size has to increase, but with every added student, a new parameter (his or her ␪-value) is added to the problem, such that the number of parameters grows at the same rate as the sample size.

Correction formulae exist to correct for this inconsistency, but they apply only to complete designs. In incomplete designs there is no general correction formula. And there is even more bad news: when using likelihood maximization,

a number of theoretical results are available to deduce good estimates of the standard errors of the estimates and to construct valid statistical tests of goodness- of-fit, but these results do not apply in the fixed-effects Rasch model. In general, therefore, it is not known to what extent reported standard errors or results of statistical tests are to be trusted if they are based on this method of estimation. Nevertheless, this method of estimation is still quite popular and is used, for example, in the programs WINSTEPS and FACETS.

Although there are few results from this method of estimation for other models than the Rasch model, it is to be expected that this kind of inconsistency will occur in other models as well. Therefore, it is advisable to avoid this method. The method of estimation described here is known in the literature as the unconditional maximum likelihood (UML) or joint maximum likelihood (JML) method of estimation. The latter name is the most used nowadays.