Skip to main content

Section 4.6 Bayes’ Theorem

Conditional probabilities 4.5.3 can be computed using the methods developed above if the appropriate information is available. Some times you will however have some information available, such as \(P(A | B)\) but need \(P(B | A)\text{.}\) The ability to "play around with history" by switching what has been presumed to occur leads to an important result known as Bayes’ Theorem.
Notice, by the definition of conditional probability 4.5.3 and the multiplication rule 4.5.5
\begin{equation*} P(S_j | A) = \frac{P(S_j \cap A)}{P(A)} = \frac{P(S_j)P( A | S_j)}{P(A)}. \end{equation*}
But using the disjointness of the partition
\begin{align*} P(A) & = P( (A \cap S_1) \cup (A \cup S_2) \cup ... \cup (A \cup S_m) )\\ & = P(A \cap S_1) + P(A \cup S_2) + ... + P(A \cup S_m)\\ & = P(S_1 \cap A) + P(S_2 \cup A) + ... + P(S_m \cup A)\\ & = P(S_1) P(A | S_1) + P(S_2)P(A | S_2) + ... + P(S_m)P(A | S_m)\\ & = \sum_{k=1}^m P(S_k)P(A | S_k) \end{align*}
Put these two expansions together to obtain the desired result.
To illustrate this result, from the web site http://stattrek.com/probability/bayes-theorem.aspx consider the following problem:
Marie is getting married tomorrow, at an outdoor ceremony in the desert. In recent years, it has rained only 5 days each year. Unfortunately, the weatherman has predicted rain for tomorrow. When it actually rains, the weatherman correctly forecasts rain 90% of the time. When it doesn’t rain, he incorrectly forecasts rain 10% of the time. What is the probability that it will rain on the day of Marie’s wedding?
Notice, all days can be classified into one of two disjoint options:
  • Rainy, in which case we can deduce from the given info that P(Rain) = 5/365
  • Not Rainy, and since this is the complement of above, P(Not Rain) = 360/365
In the notation of Bayes Theorem 4.6.1, let A represent a forecast of Rain and note you have
\begin{equation*} P(\text{Rain}) = P(S_1) = \frac{5}{365} \end{equation*}
and
\begin{equation*} P(\text{Not Rain}) = P(S_2) = \frac{360}{365}. \end{equation*}
Further, you are given the conditional probabilities
\begin{equation*} P(\text{ Forecast Rain | Rain}) = P( A | S_1) = 0.9 \end{equation*}
\begin{equation*} P(\text{ Forecast Rain | Not Rain}) = P( A | S_2) = 0.1 \end{equation*}
Notice that the question provided requests that you find the probability of Rain given that the weatherman has forecasted rain. What is given on the other hand is the reverse of that conditional probability. Using Bayes’ Theorem allows you to turn this around...
\begin{align*} P(\text{Rain}) & = P(S_1) P( A | S_1) + P(S_2) P(A | S_2)\\ & = \frac{5}{365} \cdot 0.9 + \frac{360}{365} \cdot 0.1 \end{align*}
Hence, putting these together gives
\begin{align*} P(\text{Rain | Forecast Rain}) & = \frac{\frac{5}{365} \cdot 0.9}{\frac{5}{365} \cdot 0.9 + \frac{360}{365} \cdot 0.1}\\ & = \frac{5 \cdot 0.9}{5 \cdot 0.9 + 360 \cdot 0.1}\\ & = \frac{45}{45+360} \approx 0.111 \end{align*}
So, normally there is only approximately a 1.369 percent chance of rain (5/365) on a given day but given that the weatherman has forecast rain, the chance of rain increases to a little more than 11 percent.
Here’s a standard and yet surprising example:
Your neighbor has 2 children. You learn that he has a daughter Anna. What is the probability that Anna’s sibling is a sister? Your first reaction might be that it is obviously 1/2 since (ok, roughly) it is equally likely that the other sibling was born female or male. But is that accurate?
If you were to list all of the possible female/male outcomes when having two children, then the sample space is S = {FF, FM, MF, MM} with FM meaning the first child is female and the second is male. Assuming again that girls and boys are equally likely to be born, these 4 outcomes have equal probability of \(1/4\text{.}\)
The question asks is whether the neighbor has another daughter in the set Fem2 = {FF} but since Anna is a girl then the possible outcomes now is only Fem = {FF, FM, MF}.
So,
\begin{align*} P(Fem2|Fem) & = \frac{P(Fem2 \cap Fem)}{P(Fem)} \\ & = \frac{P(FF)}{P(FF or FM or MF)}\\ & = \frac{\frac{1}{4}}{\frac{3}{4}} = \frac{1}{3} \end{align*}
Bayes’ Theorem 4.6.1 also works well in analyzing "Let’s Make A Deal" finale choice.
A biomedical research company produces \(46 \%\) of its insulin at a plant in Kansas City, and the remainder is produced at a plant in Jefferson City. Quality control has shown that \(0.6 \%\) of the insulin produced at the plant in Kansas City is defective, while \(0.8 \%\) of the insulin produced at the plant in Jefferson City is defective. What is the probability that a randomly chosen unit of insulin came from the plant in Jefferson City given that it is defective?
(Hint: Draw a tree diagram first)
Answer.
\(0.610169\)
You have to be careful to extract the conditional probabilities from the problem.
Data from Office on Smoking and Health, Centers for Disease Control and Prevention, indicate that 42% of adults who did not finish high school, 34% of high school graduates, 26% of adults who completed some college, and 13% of college graduates smoke. Suppose that one individual is selected at random and it is discovered that the individual smokes. Use the probabilities in the following table to calculate the probability that the individual is a college graduate.
Education Employed Unemployed
Not a high school graduate 0.0975 0.0080
High school graduate 0.3108 0.0128
Some college, no degree 0.1785 0.0062
Associate Degree 0.0849 0.0023
Bachelor Degree 0.1959 0.0041
Advanced Degree 0.0975 0.0015
Probability =
Hints: This problem has all the information you need, but not in the typical ready-to-use form. The table above can tell you the proportion of people with various levels of education in the population. Keep in mind that any degree (Associate, Bachelor, or Advanced) counts as graduating from college.
Answer.
\(0.198786832540129\)
Notice that having the data expressed in tabular form sometimes makes it easier to deal with.
The interactive cell below can be used to easily compute all of the conditional probabilities associated with Bayes’ Theorem 4.6.1. Notice how the relative size of the pie-shaped partition changes when you presume that an event in the space has already occurred.
You can actually also use Bayes’ Theorem 4.6.1 to answer a easy question! Indeed, suppose that you draw one card from a shuffled and standard 52 card deck. Given that you know the card is an Ace, what is the probability that it is also a Heart.
Using the Bayes’ approach, let’s break up the world into Hearts (H) and non-Hearts (N). Easily,
\begin{equation*} P(A|H) = 1/13 \end{equation*}
\begin{equation*} P(A|N) = 3/39 \end{equation*}
and so by Bayes’
\begin{equation*} P(H|A) = \frac{P(H) P(A|H)}{P(H) P(A|H) + P(N) P(A|N)} = \frac{\frac{13}{52} \cdot \frac{1}{13}}{\frac{13}{52} \cdot \frac{1}{13} + \frac{39}{52} \cdot \frac{3}{39}} = \frac{1}{4} \end{equation*}
as expected!
Your automobile insurance company uses past history to determine how to set rates by measuring the number of accidents caused by clients in various age ranges. The following table summarizes the proportion of those insured and the corresponding probabilities by age range:
Table 4.6.5. Age vs Accident Likelihood
Age Proportion of Insured Probability of Accident
16-20 0.05 0.08
21-25 0.06 0.07
26-55 0.49 0.02
55-65 0.25 0.03
over 65 0.15 0.04
One of your family friends insured by this company has an accident.
  1. Determine the conditional probability that the driver was in the 16-20 age range.
  2. Compare this to the probability that the driver was in the 18-20 age range. Discuss the difference.
  3. Determine how much more the company should charge for someone in the 16-20 age range compared to someone in the 26-55 age range.
Solution.
Plug the middle column into the first input box and the right column into the second input box of the Bayes Sage Cell.
Congratulations...your family is having a baby! As part of the prenatal care, some testing is part of the normal procedure including one for spinal bifida (which is a condition in which part of the spinal cord may be exposed.) Indeed, measurement of maternal serum AFP values is a standard tool used in obstetrical care to identify pregnancies that may have an increased risk for this disorder. You want to make plans for the new child’s care and want to know how serious to take the test results. However, some times the test indicates that the child has the disorder when in actuality it does not (a false positive) and likewise may indicate that the child does not have the disorder when in fact it does (a false negative.)
The combined accuracy rate for the screen to detect the chromosomal abnormalities mentioned above is approximately 85% with a false positive rate of 5%. This means that (from americanpregnancy.org 1 )
  • Approximately 85 out of every 100 babies affected by the abnormalities addressed by the screen will be identified. (Positive Positive)
  • Approximately 5% of all normal pregnancies will receive a positive result or an abnormal level. (False Positive)
  1. Given that your test came back negative, determine the likelihood that the child will actually have spinal bifida.
  2. Given that your test came back negative, determine the likelihood that the child will not have spina bifida
  3. Given that a positive test means you have a 1/100 to 1/300 chance of experiencing one of the abnormalities, determine the likelihood of spinal bifida in a randomly selected child.
You can get some help checking your arithmetic using the Bayes’ Sage interact.
americanpregnancy.org/prenatal-testing/first-trimester-screen/