MA40189: Topics in Bayesian statistics

Question 1

(Source: Peter M. Lee, Bayesian Statistics: an introduction, Fourth Edition, 2012)

Whether certain mice are black or brown depends upon a pair of genes, each of which is either $B$ or $b$. If both members of the pair are alike, the mouse is said to be homozygous, and if they are different it is said to be heterozygous. The mouse is brown only if it is homozygous $bb$. The offspring of a pair of mice have two such genes, one from each parent, and if the parent is heterozygous, the inherited gene is equally likely to be $B$ or $b$. Suppose that a black mouse results from a mating between two heterozygotes.

1. What are the probabilities that this mouse is homozygous and that it is heterozygous?
   
   As the two mated mice are heterozygotes then they are both $Bb$. Let $\mbox{G0}$ denote the event that the parents are both $Bb$ and $\mbox{G1-BB}$, $\mbox{G1-Bb}$ and $\mbox{G1-bb}$ respectively denote the three events that the offspring is $BB$, $Bb$ and $bb$. Let $\mbox{G1-black}$ denote the event that the offspring is black and $\mbox{G1-brown}$ that it is brown, so $\mbox{G1-black} = \mbox{G1-BB } \cup \mbox{ G1-Bb}$ and $\mbox{G1-brown} = \mbox{G1-bb}$. As the inherited gene from a heterozygous parent is equally likely to be $B$ or $b$ we have $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G0}) \ = \ \frac{1}{4}, \ P(\mbox{G1-Bb} \, | \, \mbox{G0}) \ = \ \frac{1}{2}, \ P(\mbox{G1-bb} \, | \, \mbox{G0}) \ = \ \frac{1}{4} \end{eqnarray*}$$ so that $$\begin{eqnarray*} P(\mbox{G1-black} \, | \, \mbox{G0}) & = & P(\mbox{G1-BB} \, | \, \mbox{G0}) + P(\mbox{G1-Bb} \, | \, \mbox{G0}) \ = \ \frac{3}{4}, \\ P(\mbox{G1-brown} \, | \, \mbox{G0}) & = & P(\mbox{G1-bb} \, | \, \mbox{G0}) \ = \ \frac{1}{4}. \end{eqnarray*}$$ Now, using conditional probability we have $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G1-black}, \mbox{G0}) & = & \frac{P(\mbox{G1-BB}, \mbox{G1-black} \, | \, \mbox{G0})}{P(\mbox{G1-black} \, | \, \mbox{G0})} \\ & = & \frac{P(\mbox{G1-BB} \, | \, \mbox{G0})}{P(\mbox{G1-black} \, | \, \mbox{G0})} \\ & = & \frac{\frac{1}{4}}{\frac{3}{4}} \ = \ \frac{1}{3}. \end{eqnarray*}$$ It follows immediately, as we know that the mouse is black, that $$\begin{eqnarray*} P(\mbox{G1-Bb} \, | \, \mbox{G1-black}, \mbox{G0}) & = & 1 - P(\mbox{G1-BB} \, | \, \mbox{G1-black}, \mbox{G0}) \ = \ \frac{2}{3}. \end{eqnarray*}$$ Letting $\mbox{G1-homozygous}$ denote the event that the offspring is homozygous and $\mbox{G1-heterozygous}$ that it is heterozygous we have $$\begin{eqnarray*} P(\mbox{G1-homozygous} \, | \, \mbox{G1-black}, \mbox{G0}) & = & P(\mbox{G1-BB} \, | \, \mbox{G1-black}, \mbox{G0}) \ = \ \frac{1}{3}, \\ P(\mbox{G1-heterozygous} \, | \, \mbox{G1-black}, \mbox{G0}) & = & P(\mbox{G1-Bb} \, | \, \mbox{G1-black}, \mbox{G0}) \ = \ \frac{2}{3}. \end{eqnarray*}$$

Now suppose that this mouse is mated with a brown mouse, resulting in seven offspring, all of which turn out to be black.

2. Use Bayes’ Theorem to find the probability that the black mouse was homozygous $BB$.
   
   Let $\mbox{G1}$ denote the event that the black mouse is mated with a brown mice and $\mbox{G2-7BO}$ the event that the seven offspring of $\mbox{G1}$ are black. Then, assuming that each offspring is independent, $$\begin{eqnarray*} P(\mbox{G2-7BO} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0}) & = & 1, \\ P(\mbox{G2-7BO} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0}) & = & \left(\frac{1}{2}\right)^{7}. \end{eqnarray*}$$ These can be viewed as likelihoods. Using the theorem of total probability, $$\begin{eqnarray*} P(\mbox{G2-7BO} \, | \, \mbox{G1}, \mbox{G0}) & = & P(\mbox{G2-7BO} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G1}, \mbox{G0}) + \\ & & P(\mbox{G2-7BO} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0})P(\mbox{G1-Bb} \, | \, \mbox{G1}, \mbox{G0}) \\ & = & P(\mbox{G2-7BO} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G1-black}, \mbox{G0}) + \\ & & P(\mbox{G2-7BO} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0})P(\mbox{G1-Bb} \, | \, \mbox{G1-black}, \mbox{G0}) \\ & = & 1\left(\frac{1}{3}\right) + \left(\frac{1}{2}\right)^{7}\left(\frac{2}{3}\right) \\ & = & \frac{1}{3}\left(1 + \frac{1}{2^{6}}\right). \end{eqnarray*}$$ Thus, using Bayes’ theorem we find $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G2-7BO}, \mbox{G1}, \mbox{G0}) & = & \frac{P(\mbox{G2-7B0} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G1}, \mbox{G0})}{P(\mbox{G2-7BO} \, | \, \mbox{G1}, \mbox{G0})} \\ & = & \frac{1\left(\frac{1}{3}\right)}{\frac{1}{3}\left(1 + \frac{1}{2^{6}}\right)} \\ & = & \frac{2^{6}}{2^{6} + 1} \ = \ \frac{64}{65}. \end{eqnarray*}$$ Note the form of this equation. We can view $P(\mbox{G1-BB} \, | \, \mbox{G1}, \mbox{G0})$ as our prior probability for the event G1-BB, $P(\mbox{G2-7B0} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})$ our likelihood (the probability of observing the seven black offspring given that we knew G1-BB) and $P(\mbox{G1-BB} \, | \, \mbox{G2-7BO}, \mbox{G1}, \mbox{G0})$ our posterior probability for the event G1-BB. We shall make explicit use of this relationship in the next part of the question.

3. Recalculate the same probability by regarding the seven offspring as seven observations made sequentially, treating the posterior after each observation as the prior for the next.
   
   Let $\mbox{G2-BOi}$ denote the event that the $i$th offspring is black and $\mbox{G2-iBO}$ denote the event that there have been $i$ black offspring. Thus, $\mbox{G2-iBO} = \mbox{G2-BOi} \cap \mbox{G2-(i-1)BO}$. After observing $(i-1)$ offspring, the probability for the event $\mbox{G1-BB}$ is $P(\mbox{G1-BB} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})$ whilst the probability for the event $\mbox{G1-Bb}$ is $P(\mbox{G1-Bb} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})$. Given the ancestral history, we assume that the offspring are independent. In particular, for each $i = 1, \ldots, 7$, we have the likelihoods $$\begin{eqnarray*} P(\mbox{G2-BOi} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1-BB}, \mbox{G1}, \mbox{G0}) \ = \ P(\mbox{G2-BOi} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})\ = \ 1, \\ P(\mbox{G2-BOi} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0}) \ = \ P(\mbox{G2-BOi} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0}) \ = \ \frac{1}{2}. \end{eqnarray*}$$ Then, using Bayes’ theorem we have $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & = & P(\mbox{G1-BB} \, | \, \mbox{G2-B0i}, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0}) \\ & = & \frac{P(\mbox{G2-B0i} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})}{P(\mbox{G2-BOi} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})}, \\ P(\mbox{G1-Bb} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & = & P(\mbox{G1-Bb} \, | \, \mbox{G2-B0i}, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0}) \\ & = & \frac{P(\mbox{G2-B0i} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0})P(\mbox{G1-Bb} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})}{P(\mbox{G2-BOi} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})} \end{eqnarray*}$$ Thus, noting that the denominator $P(\mbox{G2-BOi} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0})$ is common to both terms, $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & \propto & P(\mbox{G2-B0i} \, | \, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0}), \\ P(\mbox{G1-Bb} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & \propto & P(\mbox{G2-B0i} \, | \, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0})P(\mbox{G1-Bb} \, | \, \mbox{G2-(i-1)BO}, \mbox{G1}, \mbox{G0}) \end{eqnarray*}$$ which are of the form “Posterior $\propto$ Likelihood $\times$ Prior”. The constant of proportionality can be found by noting $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) + P(\mbox{G1-Bb} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) \ = \ 1. \end{eqnarray*}$$ We now add the data about the seven offspring sequentially. Our initial priors are $P(\mbox{G1-BB} \, | \, \mbox{G1-black}, \mbox{G0}) = \frac{1}{3}$ for G1-BB and $P(\mbox{G1-Bb} \, | \, \mbox{G1-black}, \mbox{G0}) = \frac{2}{3}$ for G1-Bb. We now update these beliefs after observing the event G2-1BO. We have: $$\begin{eqnarray*} \begin{array}{c|cccc} \mbox{Event} & \mbox{Prior} & \mbox{Likelihood} & \mbox{Likelihood $\times$ Prior} & \mbox{Posterior} \\ \hline \mbox{G1-BB} & \frac{1}{3} & 1 & \frac{1}{3} & \frac{\frac{1}{3}}{\frac{1}{3} + \frac{1}{3}} = \frac{1}{2} \\ \mbox{G1-Bb} & \frac{2}{3} & \frac{1}{2} & \frac{1}{3} & \frac{\frac{1}{3}}{\frac{1}{3} + \frac{1}{3}} = \frac{1}{2} \end{array} \end{eqnarray*}$$ We now add the second black offspring so that we want our beliefs after observing the event G2-2BO. We use our posteriors from the previous stage as our new priors, that is $P(\mbox{G1-BB} \, | \, \mbox{G2-1BO}, \mbox{G1}, \mbox{G0}) = \frac{1}{2}$ and $P(\mbox{G1-Bb} \, | \, \mbox{G2-1BO}, \mbox{G1}, \mbox{G0}) = \frac{1}{2}$ act as our priors. We have $$\begin{eqnarray*} \begin{array}{c|cccc} \mbox{Event} & \mbox{Prior} & \mbox{Likelihood} & \mbox{Likelihood $\times$ Prior} & \mbox{Posterior} \\ \hline \mbox{G1-BB} & \frac{1}{2} & 1 & \frac{1}{2} & \frac{\frac{1}{2}}{\frac{1}{2} + \frac{1}{4}} = \frac{2}{3} \\ \mbox{G1-Bb} & \frac{1}{2} & \frac{1}{2} & \frac{1}{4} & \frac{\frac{1}{4}}{\frac{1}{2} + \frac{1}{4}} = \frac{1}{3} \end{array} \end{eqnarray*}$$ We can continue in this vein. A straightforward inductive argument shows that $$\begin{eqnarray*} P(\mbox{G1-BB} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & = & \frac{2^{i-1}}{1 + 2^{i-1}}, \\ P(\mbox{G1-BB} \, | \, \mbox{G2-iBO}, \mbox{G1}, \mbox{G0}) & = & \frac{1}{1 + 2^{i-1}}. \end{eqnarray*}$$ Adding the seventh black offspring thus gives $$\begin{eqnarray*} \begin{array}{c|cccc} \mbox{Event} & \mbox{Prior} & \mbox{Likelihood} & \mbox{Likelihood $\times$ Prior} & \mbox{Posterior} \\ \hline \mbox{G1-BB} & \frac{32}{33} & 1 & \frac{32}{33} & \frac{\frac{32}{33}}{\frac{32}{33} + \frac{1}{66}} = \frac{64}{65} \\ \mbox{G1-Bb} & \frac{1}{33} & \frac{1}{2} & \frac{1}{66} & \frac{\frac{1}{66}}{\frac{32}{33} + \frac{1}{66}} = \frac{1}{65} \end{array} \end{eqnarray*}$$ We now adopt this explicit “Posterior $\propto$ Likelihood $\times$ Prior” approach for the answers to parts a and b.
   
   Part a. Learning that G1-black. $$\begin{eqnarray*} \begin{array}{c|cccc} \mbox{Event} & \mbox{Prior} & \mbox{Likelihood} & \mbox{Likelihood $\times$ Prior} & \mbox{Posterior} \\ \hline \mbox{G1-BB} & \frac{1}{4} & 1 & \frac{1}{4} & \frac{\frac{1}{4}}{\frac{1}{4} + \frac{1}{2} + 0} = \frac{1}{3} \\ \mbox{G1-Bb} & \frac{1}{2} & 1 & \frac{1}{2} & \frac{\frac{1}{2}}{\frac{1}{4} + \frac{1}{2}+0} = \frac{2}{3} \\ \mbox{G1-bb} & \frac{1}{4} & 0 & 0 & 0 \end{array} \end{eqnarray*}$$ Part b. Learning that G1-black mates with a brown mouse, resulting in seven black offspring. We can remove the event G1-bb as this has zero prior probability. $$\begin{eqnarray*} \begin{array}{c|cccc} \mbox{Event} & \mbox{Prior} & \mbox{Likelihood} & \mbox{Likelihood $\times$ Prior} & \mbox{Posterior} \\ \hline \mbox{G1-BB} & \frac{1}{3} & 1 & \frac{1}{3} & \frac{\frac{1}{3}}{\frac{1}{3} + \frac{1}{3}\left(\frac{1}{2}\right)^{6}} = \frac{64}{65} \\ \mbox{G1-Bb} & \frac{2}{3} & \left(\frac{1}{2}\right)^{7} & \frac{1}{3}\left(\frac{1}{2}\right)^{6} & \frac{\frac{1}{3}\left(\frac{1}{2}\right)^{6}}{\frac{1}{3} + \frac{1}{3}\left(\frac{1}{2}\right)^{6}} = \frac{1}{65} \\ \end{array} \end{eqnarray*}$$

4. What is the probability that an eighth offspring is also black.
   
   Here we are looking at the predictive distribution. Let G2-BO8 denote the event that the eighth offspring is black. The solution is once again to use total probability with the two events G1-BB and G1-Bb. We have $$\begin{eqnarray*} P(\mbox{G2-BO8} \, | \, \mbox{G2-7BO}, \mbox{G1}, \mbox{G0}) & = & P(\mbox{G2-BO8} \, | \, \mbox{G2-7B0}, \mbox{G1-BB}, \mbox{G1}, \mbox{G0})P(\mbox{G1-BB} \, | \, \mbox{G2-7BO}, \mbox{G1}, \mbox{G0}) + \\ & & P(\mbox{G2-BO8} \, | \, \mbox{G2-7B0}, \mbox{G1-Bb}, \mbox{G1}, \mbox{G0})P(\mbox{G1-Bb} \, | \, \mbox{G2-7B0}, \mbox{G1}, \mbox{G0}) \\ & = & 1\left(\frac{64}{65}\right) + \frac{1}{2}\left(\frac{1}{65}\right) \ = \ \frac{129}{130}. \end{eqnarray*}$$