The Runs Test

5.1.1 The Runs Test

The runs test assesses whether or not a sequence of observations can be accepted as a random sequence, that is, with independent successive observations. Note that most tests of hypotheses do not care about the order of the observations. Consider, for instance, the meteorological data used in Example 4.1. In this example, when testing the mean based on a sample of maximum temperatures, the order of the observations is immaterial. The maximum temperatures could be ordered by increasing or decreasing order, or could be randomly shuffled, still giving us exactly the same results.

Sometimes, however, when analysing sequences of observations, one has to decide whether a given sequence of values can be assumed as exhibiting a random behaviour.

Consider the following sequences of n = 12 trials of a dichotomous experiment, as one could possibly obtain when tossing a coin:

Sequence 1: 000000111111 Sequence 2:

010101010101 Sequence 3:

Sequences 1 and 2 would be rejected as random since a dependency pattern is 1 clearly present . Such sequences raise a reasonable suspicion concerning either the “fairness” of the coin-tossing experiment or the absence of some kind of data manipulation (e.g. sorting) of the experimental results. Sequence 3, on the other hand, seems a good candidate of a sequence with a random pattern.

The runs test analyses the randomness of a sequence of dichotomous trials. Note that all the tests described in the previous chapter (and others to be described next as well) are insensitive to data sorting. For instance, when testing the mean of the

three sequences above, with H 0 : µ = 6/12 = ½, one obtains the same results. The test procedure uses the values of the number of occurrences of each category, say n 1 and n 2 for 1 and 0 respectively, and the number of runs, i.e., the number of occurrences of an equal value subsequence delimited by a different value. For sequence 3, the number of runs, r, is equal to 7, as seen below:

Sequence 3: 001011101100 Runs:

Note that we are assessing the randomness of the sequence, not of the process that generated it.

5.1 Inference on One Population 173

The runs test assesses the null hypothesis of sequence randomness, using the sampling distribution of r, given n 1 and n 2 . Tables of this sampling distribution can

be found in the literature. For large n 1 or n 2 (say above 20) the sampling distribution of r is well approximated by the normal distribution with the following parameters:

5.2 ( n 1 + n 2 )

Notice that the number of runs always satisfies, 1 ≤ r ≤ n, with n = n 1 +n 2 . The null hypothesis is rejected when there are either too few runs (as in Sequence 1) or too many runs (as in Sequence 2). For the previous sequences, at a 5% level the

critical values of r for n 1 =n 2 = 6 are 3 and 11, i.e. the non-critical region of r is [4, 10]. We, therefore, reject at 5% level the null hypothesis of randomness for Sequence 1 (r = 2) and Sequence 2 (r = 12), and do not reject the null hypothesis for Sequence 3 (r = 7).

The runs test can be used with any sequence of values and not necessarily dichotomous, if previously the values are dichotomised, e.g. using the mean or the median.

Example 5.1

Q: Consider the noise sequence in the Signal & Noise dataset (first column) generated with the “normal random number” routine of EXCEL with zero mean. The sequence has n = 100 noise values. Use the runs test to assess the randomness of the sequence.

A: We apply the SPSS runs test command, using an imposed ( Custom) dichotomization around zero, obtaining an observed two-tailed significance of p = 0.048. At a 5% level of significance the randomness of the sequence is not rejected. We may also use the MATLAB or R runs function. We obtain the values of Table 5.1. The interval [n low ,n up ] represents the non critical region. We see that the observed number of runs coincides with one of the interval ends.

Table 5.1. Results obtained with MATLAB or R runs test for the noise data.

Q: Consider the Forest Fires dataset (see Appendix E), which contains the area (ha) of burnt forest in Portugal during the period 1943-1978. Is there evidence from this sample, at a 5% significance level, that the area of burnt forest behaves as

a random sequence?

174 5 Non-Parametric Tests of Hypotheses

A: The area of burnt forest depending on the year is shown in Figure 5.1. Notice that there is a clear trend we must remove before attempting the runs test. Figure

5.1 also shows the regression line with a null intercept, i.e. passing through the point (0,0), obtained with the methods that will be explained later in Chapter 7. We now compute the deviations from the linear trend and use them for the runs test. When analysed with SPSS, we find an observed two-tailed significance of p = 0.335. Therefore, we do not reject the null hypothesis that the area of burnt forest behaves as a random sequence superimposed on a linear trend.

25000 Area (ha)

Figure 5.1. Area of burnt forest in Portugal during the years 1943-1978. The dotted line is a linear fit with null intercept.

Commands 5.1. SPSS, MATLAB and R commands used to perform the runs test.

SPSS Analyze; Nonparametric Tests; Runs MATLAB

runs(x,alpha) R

runs(x,alpha=0.05)

STATISTICA, MATLAB statistical toolbox and R stats package do not have the runs test. We provide the runs function for MATLAB and R (see appendix F)

returning the values of Table 5.1. The function should only be used when n 1 or n 2 are large (say, above 20).