Measures of Association Revisited

5.2.4 Measures of Association Revisited

When analysing contingency tables, it is also convenient to assess the degree of association between the variables, using the ordinal and nominal association measures described in sections 2.3.5 and 2.3.6, respectively. As in 4.4.1, the

198 5 Non-Parametric Tests of Hypotheses hypotheses in a two-sided test concerning any measure of association γ are

formalised as:

H 0 : γ = 0;

H 1 : γ ≠ 0.

5.2.4.1 Measures for Ordinal Data

Let X and Y denote the variables whose association is being assessed. The exact

values of the sampling distribution of the Spearman’s rank correlation, when H 0 is

true, can be derived if we note that for any given ranking of Y, any rank order of X is equally likely, and vice-versa. Therefore, any particular ranking has a probability of occurrence of 1/n!. As an example, let us consider the situation of n = 3, with X and Y having ranks 1, 2 and 3. As shown in Table 5.14, there are 3! = 6 possible permutations of the X ranks. Applying formula 2.21, one then obtains the r s values

shown in the last row. Therefore, under H 0 , the ±1 values have a 1/6 probability

and the ±½ values have a 1/3 probability. When n is large (say, above 20), the

significance of r s under H 0 can be obtained using the test statistic:

z * = r s n − 1 , 5.25

which is approximately distributed as the standard normal distribution.

Table 5.14. Possible rankings and Spearman correlation for n = 3.

r s 1 0.5 0.5 −0.5

In order to test the significance of the gamma statistic a large sample (say, above 25) is required. We then use the test statistic:

which, under H 0 γ = 0), is approximately distributed as the standard normal ( distribution. The values of P and Q were defined in section 2.3.5. The Spearman correlation and the gamma statistic were computed for Example

5.12, with the results shown in Table 5.15. We see that the observed significance is

5.2 Contingency Tables 199

very low, leading to the conclusion that there is an association between both variables (PERF, PROG).

Table 5.15. Measures of association for ordinal data computed with SPSS for Example 5.12.

Asymp. Std.

Value Approx. T Approx. Sig.

Error

Gamma

0.076 5.458 0.000 Spearman Correlation 0.332

5.2.4.2 Measures for Nominal Data

In Chapter 2, the following measures of association were described: the index of association (phi coefficient), the proportional reduction of error (Goodman and

Kruskal lambda), and the κ statistic for the degree of agreement. Note that taking into account formulas 2.24 and 5.20, the phi coefficient can be computed as:

5.27 n

with the phi coefficient now lying in the interval [0, 1]. Since the asymptotic distribution of T 1 is the standard normal distribution, one can then use this distribution in order to evaluate the significance of the signed phi coefficient (using

the sign of O 11 O 22 − O 12 O 21 ) multiplied by n .

Table 5.16 displays the value and significance of the phi coefficient for Example

5.9. The computed two-sided significance of phi is 0.083; therefore, at a 5% significance level, we do not reject the hypothesis that there is no association between SEX and INIT.

Table 5.16. Phi coefficient computed with SPSS for the Example 5.9 with the two- sided significance.

Value Approx. Sig. Phi 0.151

The proportional reduction of error has a complex sampling distribution that we will not discuss. For Example 5.9 the only situation of interest for this measure of association is: INIT depending on SEX. Its value computed with SPSS is 0.038. This means that variable SEX will only reduce by about 4% the error of predicting

200 5 Non-Parametric Tests of Hypotheses

INIT. As a matter of fact, when using INIT alone, the prediction error is (131 – 121)/131 = 0.076. With the contribution of variable SEX, the prediction error is the same (5/131 + 5/131). However, since there is a tie in the row modes, the contribution of INIT is computed as half of the previous error.

In order to test the significance of the κ statistic measuring the agreement among several variables, the following statistic, approximately normally distributed for large n with zero mean and unit standard deviation, is used:

z = κ / var () κ , with

var () κ ≈ ∑ . 5.28a

2 2 3 P ()(

E 2 κ 3 )() [ P E ] + 2 ( κ − 2 ) p j

P 2 ()

As described in 2.3.6.3, the κ statistic can be computed with function kappa implemented in MATLAB or R; kappa(x,alpha)computes for a matrix x, (formatted as columns N, S and P in Table 2.13), the row vector denoted [ko,z,zc] in MATLAB containing the observed value of κ, ko, the z value of formula 5.28 and the respective critical value, zc, at alpha level. The meaning of the returned values for the R kappa function is the same. The results of the κ statistic significance for Example 2.11 are obtained as shown below. We see that the null hypothesis (disagreement among all four classifiers) is rejected at a 5% level of significance, since z > zc.

[ko,z,zc]=kappa(x,0.05) ko =

0.2130 z= 3.9436 zc = 3.2897