The sampling distribution of the mean

5.1 The sampling distribution of the mean

  We can now use the properties of expected values and variances to derive the expected value and variance of the sample mean .

  thus the expected value of the sample mean is the actual population mean of the random variables. A consequence of this is that

  2 E[( ¯ 2 X

  n − µ) ] = E[( ¯ X n − E( ¯ X n )) ] = Var( ¯ X n ) (5.7)

  This takes us directly to our goal. Using the properties of the variance, and con-

  sidering that the random variables X 1 ,...,X n are independent we get

  1 n σ 2

  V ar( ¯ X n )=

  Var(X i )=

  n 2

  n

  i=1

  5.1. THE SAMPLING DISTRIBUTION OF THE MEAN

  The standard deviation of the mean is easier to interpret than the variance of the mean. It roughly represents how much the means of independent randomly ob- tained samples typically differ from the population mean. Since the standard de- viation of the mean is so important, many statisticians give it a special name: the standard error of the mean. Thus the standard error of the mean simply is the

  square root of the variance of the mean, and it is represented as Sd( ¯ X n ) or Se( ¯ X n ).

  Finally we are done! Equation 5.9 tells us that the uncertainty of the sample mean (its standard deviation) increases proportionally to the uncertainty about individual observations (σ) and decreases proportionally to the square root of the number of observations. Thus, if we want to double the precision of the sample mean (reduce its standard deviation by half) we need to quadruple the number of observations in the sample.

  Example: Let X 1 and X 2 be independent Bernoulli random variables with pa-

  rameter µ = 0.5. Thus E(X 1 ) = E(X 2 ) = µ = 0.5 and Var(X 1 ) = Var(X 2 )=

  σ 2 = 0.25, moreover

  E( ¯ X 2 ) = (0)(0.25) + (0.5)(0.5) + (1)(0.25) = 0.5 = µ

  Var( ¯ 2 X

  2 )=σ 2 = 0.125

  5.1.1 Central Limit Theorem

  This is a very important theorem which we will not prove here. The theorem tells

  us that as n goes to infinity, the cumulative distribution of ¯ X n is closely approx- imated by that of a Gaussian random variable with mean E( ¯ X n ) and standard

  deviation Sd( ¯ X n ). In practice for n ≥ 30 the Gaussian cumulative distribution

  provides very good approximations. Example: We toss a fair coin 100 times. What is the probability that the pro-

  portion of “heads” be smaller or equal to 0.45? Answer: Tossing a fair coin 100 times can be modeled using 100 independent

  identically distributed Bernoulli random variables each of which has parameter

  56 CHAPTER 5. THE PRECISION OF THE ARITHMETIC MEAN

  µ = 0.5. Let’s represent these 100 random variables as X 1 ,...,X 100 where X i

  takes the value 1 if the i th time we toss the coin we get heads. Thus the proportion of heads is the average of the 100 random variables, we will represent it as ¯ X.

  X= ¯ 1 (X 1 + ···+X 100 )

  We know

  µ = E(X 1 )= · · · = E(X 100 ) = 0.5

  σ 2 = Var(X

  1 )= · · · = Var(X 100 ) = 0.25

  Var( ¯ 2 X) = σ 100 = 0.0025

  Since n = 100 > 30 the cumulative distribution of ¯ X is approximately Gaussian. Thus

  0.45 − 0.5

  P(¯ X ≤ 0.45) = F X ¯ (0.45) ≈ Φ( √

  ) = Φ( −1) = 0.1586