# Difference between revisions of "Eleisha's Segment 38: Mutual Information"

To Calculate 1. Suppose the domain of a model are the five integers $\displaystyle x = \{1,2,3,4,5\}$ , and that your proposal distribution is: "When $\displaystyle x_1 = 2,3,4$ , choose with equal probability $\displaystyle x_2 = x_1 \pm 1$ . For [/itex] x_1=1 [/itex] always choose $\displaystyle x_2 =2$ . For $\displaystyle x_1=5$ always choose $\displaystyle x_2 =4$ . What is the ratio of q's that goes into the acceptance probability $\displaystyle \alpha(x_1,x_2)$ for all the possible values of $\displaystyle x_1$ and $\displaystyle x_2$ ?

2. Suppose the domain of a model is $\displaystyle -\infty < x < \infty$ and your proposal distribution is (perversely),

$\displaystyle q(x_2|x_1) = \begin{cases}\tfrac{7}{2}\exp[-7(x_2-x_1)],\quad & x_2 \ge x_1 \\ \tfrac{5}{2}\exp[-5(x_1-x_2)],\quad & x_2 < x_1 \end{cases}$ Sketch this distribution as a function of $\displaystyle x_2-x_1$ . Then, write down an expression for the ratio of $\displaystyle q$ 's that goes into the acceptance probability $\displaystyle \alpha(x_1,x_2)$ .

1. Suppose an urn contains 7 large orange balls, 3 medium purple balls, and 5 small green balls. When balls are drawn randomly, the larger ones are more likely to be drawn, in the proportions large:medium:small = 6:4:3. You want to draw exactly 6 balls, one at a time without replacement. How would you use Gibbs sampling to learn: (a) How often do you get 4 orange plus 2 of the same (non-orange) color? (b) What is the expectation (mean) of the product of the number of purple and number of green balls drawn?

2. How would you do the same problem computationally but without Gibbs sampling?

3. How would you do the same problem non-stochastically (e.g., obtain answers to 12 significant figures)? (Hint: This is known as the Wallenius non-central hypergeometric distribution.)