Segment 39 Sanmit Narvekar
Segment 39
To Calculate
1. Suppose the domain of a model are the five integers Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x = \{1,2,3,4,5\}} , and that your proposal distribution is: "When Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1 = 2,3,4} , choose with equal probability Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2 = x_1 \pm 1} . For Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1=1} always choose Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2 =2} . For Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1=5} always choose Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2 =4} . What is the ratio of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q} 's that goes into the acceptance probability Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha(x_1,x_2)} for all the possible values of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1} and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2} ?
We ignore showing all transitions that have 0 probability under the proposal distribution. The q ratio that goes into the acceptance probability for Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha(x_1, x_2)}
is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{q(x_1|x_2)}{q(x_2|x_1)}}
. These values are given above. The resulting table looks like this:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1} | Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2} | Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q(x_1|x_2)} | Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q(x_2|x_1) } | q ratio |
---|---|---|---|---|
1 | 2 | 0.5 | 1 | 0.5 |
2 | 1 | 1 | 0.5 | 2 |
2 | 3 | 0.5 | 0.5 | 1 |
3 | 2 | 0.5 | 0.5 | 1 |
3 | 4 | 0.5 | 0.5 | 1 |
4 | 3 | 0.5 | 0.5 | 1 |
4 | 5 | 1 | 0.5 | 2 |
5 | 4 | 0.5 | 1 | 0.5 |
2. Suppose the domain of a model is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle -\infty < x < \infty}
and your proposal distribution is (perversely),
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q(x_2|x_1) = \begin{cases}\tfrac{7}{2}\exp[-7(x_2-x_1)],\quad & x_2 \ge x_1 \\ \tfrac{5}{2}\exp[-5(x_1-x_2)],\quad & x_2 < x_1 \end{cases}}
Sketch this distribution as a function of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2-x_1} . Then, write down an expression for the ratio of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q} 's that goes into the acceptance probability Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha(x_1,x_2)} .
Here is the MATLAB code to sketch the distribution as a function of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_2-x_1}
(yes I know it's not the most efficient code...):
function y = qfunc(x1, x2) if x2 >= x1 y = (7/2) * exp(-7 * (x2-x1)); else y = (5/2) * exp(-5 * (x1-x2)); end
x1 = -1:0.01:1; x2 = -1:0.01:1; x2mx1 = []; qdist = []; for a=1:numel(x1) for b=1:numel(x2) x2mx1 = [x2mx1, x2(b)-x1(a)]; qdist = [qdist, qfunc(x1(a), x2(b))]; end end [x2mx1, imap] = sort(x2mx1); qdist = qdist(imap); plot(x2mx1, qdist) xlabel('x2-x1') ylabel('q(x2|x1)')
And here is the resulting distribution sketch:
The ratio of q's that goes into the acceptance probability can be calculated as before. You have to be careful to select the right pieces and do it case by case. It's helpful to first write down Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q(x_1|x_2)}
:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle q(x_1|x_2) = \begin{cases} \tfrac{7}{2}\exp[-7(x_1-x_2)],\quad & x_1 \ge x_2 \\ \tfrac{5}{2}\exp[-5(x_2-x_1)],\quad & x_1 < x_2 \end{cases} }
Then, the ratio can be calculated easily case by case:
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \frac{q(x_1|x_2)}{q(x_2|x_1)} = \begin{cases} \tfrac{7}{5}\exp[-2(x_1-x_2)],\quad & x_1 > x_2 \\ \tfrac{5}{7}\exp[-2(x_1-x_2)],\quad & x_1 < x_2 \\ \exp[14(x_2-x_1)],\quad & x_1 = x_2 \end{cases} }
To Think About
1. Suppose an urn contains 7 large orange balls, 3 medium purple balls, and 5 small green balls. When balls are drawn randomly, the larger ones are more likely to be drawn, in the proportions large:medium:small = 6:4:3. You want to draw exactly 6 balls, one at a time without replacement. How would you use Gibbs sampling to learn: (a) How often do you get 4 orange plus 2 of the same (non-orange) color? (b) What is the expectation (mean) of the product of the number of purple and number of green balls drawn?
2. How would you do the same problem computationally but without Gibbs sampling?
3. How would you do the same problem non-stochastically (e.g., obtain answers to 12 significant figures)? (Hint: This is known as the Wallenius non-central hypergeometric distribution.)
[Answers: 0.155342 and 1.34699]