The Classic Approach

6.1 The Classic Approach

  Classic statisticians (also known as frequentists) view scientific hypotheses as propositions with a fixed truth value. Within this approach a hypothesis is either true or false and thus it makes no sense to talk about its probability. For example, for a frequentist the hypothesis “Smoking causes cancer” is either true or false. It makes no sense to say that there is a 99 chance that smoking causes can- cer. In this approach we perform experiments, gather data and if the data provide sufficient evidence against a hypothesis we reject the hypothesis.

  Here is an example that illustrates how the classic approach works in practice. Suppose we want to disprove the hypothesis that a particular coin is fair. We toss the coin 100 times and get 74 tails. We model our observations as a random variable ¯ X which is the average of 100 iid Bernoulli ravs with unknown parameter

  60 CHAPTER 6. INTRODUCTION TO STATISTICAL HYPOTHESIS TESTING

  The statement that we want to reject is called the null hypothesis. In this case the null hypothesis is that µ = 0.5, i.e., that the coin is fair. We do not know E( ¯

  X) nor Var( ¯ X). However we know what these two values would be if the null hypothesis were true. We represent the conditional expected value of ¯ X assuming

  that the null hypothesis is true as E( ¯ X |H n true). The variance of ¯ X assuming

  that the null hypothesis is true is represented as Var( ¯ X |H n true). In our example

  E( ¯ X |H n true) = 0.5

  Var( ¯ X |H n true) = 0.25100

  Moreover, due to the central limit theorem the cumulative distribution of ¯ X should

  be approximately Gaussian. We note that a proportion of 0.74 tails corresponds to the following standard score

  Using the standard Gaussian tables we find

  P( 6 {¯ X ≥ 4.8} | H

  n true) < 110

  If the null hypothesis were true the chances of obtaining 74 tails or more are less than one in a million. We now have two alternatives: 1) We can keep the null hypothesis in which case we would explain our observations as an extremely un- probable event due to chance. 2) We can reject the null hypothesis in view of the fact that if it were correct the results of our experiment would be an extremely im- probable event. Sir Ronald Fisher, a very influential classic statistician, explained these options as follows:

  The force with which such conclusion is supported logically is that of a simple disjunction: Either an exceptionally rare chance event has occurred, or the theory or random distribution [the null hypothesis] is not true [?]

  In our case obtaining 74 tails would be such a rare event if the null hypoth- esis were true that we feel compelled to reject the null hypothesis and conclude that the coin is loaded. But what if we had obtained say 55 tails? Would that

  be enough evidence to reject the null hypothesis? What should we consider as enough evidence? What standards should we use to reject a hypothesis? The next sections explain the classic approach to these questions. To do so it is convenient to introduce the concept of Type I and Type II errors.

  6.2. TYPE I AND TYPE II ERRORS