Bayes Estimates Using Decision Theory Framework

18.3 Bayes Estimates Using Decision Theory Framework

  Using Bayesian methodology, the posterior distribution of a parameter can be obtained. Bayes estimates can also be derived using the posterior distribution and

  a loss function when a loss is incurred. A loss function is a function that describes the cost of a decision associated with an event of interest. Here we only list a few commonly used loss functions and their associated Bayes estimates.

  Squared-Error Loss

  Definition 18.3: The squared-error loss function is

  L(θ, a) = (θ

  − a) 2 ,

  where θ is the parameter (or state of nature) and a an action (or estimate).

  A Bayes estimate minimizes the posterior expected loss, given on the observed sample data.

  Theorem 18.1: The mean of the posterior distribution π(θ |x), denoted by θ ∗ , is the Bayes esti-

  mate of θ under the squared-error loss function.

  Example 18.8: Find the Bayes estimates of p, for all the values of x, for Example 18.1 when the

  squared-error loss function is used. Solution : When x = 0, p ∗ = (0.1)(0.6550) + (0.2)(0.3450) = 0.1345.

  When x = 1, p ∗ = (0.1)(0.4576) + (0.2)(0.5424) = 0.1542. When x = 2, p ∗ = (0.1)(0.2727) + (0.2)(0.7273) = 0.1727.

  Note that the classical estimate of p is ˆ p = xn = 0, 12, and 1, respectively, for the x values at 0, 1, and 2. These classical estimates are very different from the corresponding Bayes estimates.

  Example 18.9: Repeat Example 18.8 in the situation of Example 18.2.

  Solution : Since the posterior distribution of p is a B(x + 1, 3 − x) distribution (see Section

  6.8 on page 201), the Bayes estimate of p is

  which yields p ∗ = 14 for x = 0, p ∗ = 12 for x = 1, and p ∗ = 34 for x = 2, respectively. Notice that when x = 1 is observed, the Bayes estimate and the classical estimate ˆ p are equivalent.

  For the normal situation as described in Example 18.5, the Bayes estimate of μ under the squared-error loss will be the posterior mean μ ∗ .

  Example 18.10: Suppose that the sampling distribution of a random variable, X, is Poisson with

  parameter λ. Assume that the prior distribution of λ follows a gamma distribution

  Chapter 18 Bayesian Statistics

  with parameters (α, β). Find the Bayes estimate of λ under the squared-error loss function.

  Solution : Using Example 18.3, we conclude that the posterior distribution of λ follows a

  gamma distribution with parameters (x + α, (1 + 1β) −1 ). Using Theorem 6.4, we obtain the posterior mean

  Since the posterior mean is the Bayes estimate under the squared-error loss, ˆ λ is our Bayes estimate.

  Absolute-Error Loss

  The squared-error loss described above is similar to the least-squares concept we discussed in connection with regression in Chapters 11 and 12. In this section, we introduce another loss function as follows.

  Definition 18.4: The absolute-error loss function is defined as

  L(θ, a) = |θ − a|,

  where θ is the parameter and a an action.

  Theorem 18.2: The median of the posterior distribution π(θ |x), denoted by θ ∗ , is the Bayes

  estimate of θ under the absolute-error loss function.

  Example 18.11: Under the absolute-error loss, find the Bayes estimator for Example 18.9 when

  x = 1 is observed. Solution : Again, the posterior distribution of p is a B(x + 1, 3 − x). When x = 1, it is a beta

  distribution with density π(p | x = 1) = 6x(1 − x) for 0 < x < 1 and 0 otherwise. The median of this distribution is the value of p ∗ such that

  which yields the answer p ∗ = 1 2 . Hence, the Bayes estimate in this case is 0.5.

  Exercises

  18.1 Estimate the proportion of defectives being pro-

  p

  duced by the machine in Example 18.1 if the random

  π(p)

  sample of size 2 yields 2 defectives.

  If 2 of the next 9 drinks from this machine overflow, find

  18.2 Let us assume that the prior distribution for the (a) the posterior distribution for the proportion p; proportion p of drinks from a vending machine that (b) the Bayes estimate of p. overflow is

  Exercises

  18.3 Repeat Exercise 18.2 when 1 of the next 4 drinks (a) a Bayes estimate of the true average daily profit for overflows and the uniform prior distribution is

  this building;

  π(p) = 10,

  0.05 < p < 0.15.

  (b) a 95 Bayesian interval of μ for this building; (c) the probability that the average daily profit from

  the machine in this building is between 24.00 and

  18.4 Service calls come to a maintenance center ac-

  cording to a Poisson process with λ calls per minute.

  A data set of 20 one-minute periods yields an average 18.9 The mathematics department of a large uni-

  of 1.8 calls. If the prior for λ follows an exponential versity is designing a placement test to be given to distribution with mean 2, determine the posterior dis- incoming freshman classes. Members of the depart- tribution of λ.

  ment feel that the average grade for this test will vary from one freshman class to another. This variation of

  18.5 A previous study indicates that the percentage the average class grade is expressed subjectively by a

  of chain smokers, p, who have lung cancer follows a normal distribution with mean μ 0 = 72 and variance

  beta distribution (see Section 6.8) with mean 70 and σ 2 standard deviation 10. Suppose a new data set col-

  lected shows that 81 out of 120 chain smokers have (a) What prior probability does the department assign lung cancer.

  to the actual average grade being somewhere be- tween 71.8 and 73.4 for next year’s freshman class?

  (a) Determine the posterior distribution of the percent-

  age of chain smokers who have lung cancer by com- (b) If the test is tried on a random sample of 100 stu- bining the new data and the prior information.

  dents from the next incoming freshman class, re- sulting in an average grade of 70 with a variance of

  (b) What is the posterior probability that p is larger

  64, construct a 95 Bayesian interval for μ.

  than 50?

  (c) What posterior probability should the department

  assign to the event of part (a)?

  18.6 The developer of a new condominium complex claims that 3 out of 5 buyers will prefer a two-bedroom

  unit, while his banker claims that it would be more 18.10 Suppose that in Example 18.7 the electrical

  correct to say that 7 out of 10 buyers will prefer a two- firm does not have enough prior information regard- bedroom unit. In previous predictions of this type, the ing the population mean length of life to be able to banker has been twice as reliable as the developer. If assume a normal distribution for μ. The firm believes,

  12 of the next 15 condominiums sold in this complex however, that μ is surely between 770 and 830 hours, are two-bedroom units, find

  and it is thought that a more realistic Bayesian ap- proach would be to assume the prior distribution

  (a) the posterior probabilities associated with the

  claims of the developer and banker;

  (b) a point estimate of the proportion of buyers who

  π(μ) =

  , 770 < μ < 830.

  prefer a two-bedroom unit.

  If a random sample of 25 bulbs gives an average life of

  18.7 The burn time for the first stage of a rocket is 780 hours, follow the steps of the proof for Example a normal random variable with a standard deviation

  18.5 to find the posterior distribution

  of 0.8 minute. Assume a normal prior distribution for

  π(μ

  |x 1 ,x 2 ,...,x 25 ).

  μ with a mean of 8 minutes and a standard deviation

  of 0.2 minute. If 10 of these rockets are fired and the

  first stage has an average burn time of 9 minutes, find 18.11 Suppose that the time to failure T of a certain

  a 95 Bayesian interval for μ.

  hinge is an exponential random variable with probabil- ity density

  18.8 The daily profit from a juice vending machine placed in an office building is a value of a normal ran-

  2 f (t) = θe −θt , dom variable with unknown mean μ and variance σ t > 0. . Of course, the mean will vary somewhat from building From prior experience we are led to believe that θ is

  to building, and the distributor feels that these average

  a value of an exponential random variable with proba-

  daily profits can best be described by a normal distri- bility density

  bution with mean μ 0 = 30.00 and standard deviation

  σ 0 = 1.75. If one of these juice machines, placed in

  π(θ) = 2e −2θ , θ > 0.

  a certain building, showed an average daily profit of ¯ x = 24.90 during the first 30 days with a standard If we have a sample of n observations on T , show that deviation of s = 2.10, find

  the posterior distribution of Θ is a gamma distribution

  Chapter 18 Bayesian Statistics

  with parameters

  18.14 A random variable X follows an exponential

  distribution with mean 1β. Assume the prior distri-

  −1

  n

  bution of β is another exponential distribution with

  α=n+1 and β=

  t i +2

  mean 2.5. Determine the Bayes estimate of β under

  i=1

  the absolute-error loss function.

  18.15 A random sample X 1 ,...,X n comes from

  18.12 Suppose that a sample consisting of 5, 6, 6, 7,

  a uniform distribution (see Section 6.1) population

  5, 6, 4, 9, 3, and 6 comes from a Poisson population U (0, θ) with unknown θ. The data are given below: with mean λ. Assume that the parameter λ follows a gamma distribution with parameters (3, 2). Under the

  squared-error loss function, find the Bayes estimate of

  λ.

  Suppose the prior distribution of θ has the density

  18.13 A random variable X follows a negative bino- mial distribution with parameters k = 5 and p [i.e.,

  ∗

  π(θ) = θ

  2 , θ > 1,