Potential Misconceptions and Hazards; Relationship to Material in Other Chapters
10.15 Potential Misconceptions and Hazards; Relationship to Material in Other Chapters
One of the easiest ways to misuse statistics relates to the final scientific conclusion drawn when the analyst does not reject the null hypothesis H 0 . In this text, we have attempted to make clear what the null hypothesis means and what the al- ternative means, and to stress that, in a large sense, the alternative hypothesis is much more important. Put in the form of an example, if an engineer is attempt-
ing to compare two gauges using a two-sample t-test, and H 0 is “the gauges are equivalent” while H 1 is “the gauges are not equivalent,” not rejecting H 0 does not lead to the conclusion of equivalent gauges. In fact, a case can be made for never writing or saying “accept H 0 ”! Not rejecting H 0 merely implies insufficient evidence. Depending on the nature of the hypothesis, a lot of possibilities are still not ruled out.
In Chapter 9, we considered the case of the large-sample confidence interval using
x−μ ¯ z= √ . s/ n
In hypothesis testing, replacing σ by s for n < 30 is risky. If n ≥ 30 and the distribution is not normal but somehow close to normal, the Central Limit Theorem is being called upon and one is relying on the fact that with n ≥ 30, s ≈ σ. Of course, any t-test is accompanied by the concomitant assumption of normality. As in the case of confidence intervals, the t-test is relatively robust to normality. However, one should still use normal probability plotting, goodness-of-fit tests, or other graphical procedures when the sample is not too small.
Most of the chapters in this text include discussions whose purpose is to relate the chapter in question to other material that will follow. The topics of estimation
10.15 Potential Misconceptions and Hazards 387 and hypothesis testing are both used in a major way in nearly all of the tech-
niques that fall under the umbrella of “statistical methods.” This will be readily noted by students who advance to Chapters 11 through 16. It will be obvious that these chapters depend heavily on statistical modeling. Students will be ex- posed to the use of modeling in a wide variety of applications in many scientific and engineering fields. It will become obvious quite quickly that the framework of a statistical model is useless unless data are available with which to estimate parameters in the formulated model. This will become particularly apparent in Chapters 11 and 12 as we introduce the notion of regression models. The concepts and theory associated with Chapter 9 will carry over. As far as material in the present chapter is concerned, the framework of hypothesis testing, P -values, power of tests, and choice of sample size will collectively play a major role. Since initial model formulation quite often must be supplemented by model editing before the analyst is sufficiently comfortable to use the model for either process understand- ing or prediction, Chapters 11, 12, and 15 make major use of hypothesis testing to supplement diagnostic measures that are used to assess model quality.