# Difference between revisions of "Eleisha's Segment 28: Gaussian Mixtures Models in 1D"

(Created page with "<b> To Calculate: </b> 1. Draw a sample of 100 points from the uniform distribution <math> U(0,1) </math> . This is your data set. Fit GMM models to your sample (now conside...") |
|||

Line 5: | Line 5: | ||

2. Multiplying a lot of individual likelihoods will often underflow. (a) On average, how many values drawn from U(0,1) can you multiply before the product underflows to zero? (b) What, analytically, is the distribution of the sum of <math> N </math> independent values <math> \log(U) </math> , where <math> U\sim U(0,1 </math> )? (c) Is your answer to (a) consistent with your answer to (b)? | 2. Multiplying a lot of individual likelihoods will often underflow. (a) On average, how many values drawn from U(0,1) can you multiply before the product underflows to zero? (b) What, analytically, is the distribution of the sum of <math> N </math> independent values <math> \log(U) </math> , where <math> U\sim U(0,1 </math> )? (c) Is your answer to (a) consistent with your answer to (b)? | ||

− | <b> To Think About </b> | + | <b> To Think About: </b> |

1. Suppose you want to approximate some analytically known function <math> f(x) </math> (whose integral is finite), as a sum of <math> K </math> Gaussians with different centers and widths. You could pretend that <math> f(x) </math> (or some scaling of it) was a probability distribution, draw <math> N </math> points from it and do the GMM thing to find the approximating Gaussians. Now take the limit <math> N\rightarrow \infty </math> , figure out how sums become integrals, and write down an iterative method for fitting Gaussians to a given <math> f(x) </math>. Does it work? (You can assume that well-defined definite integrals can be done numerically.) | 1. Suppose you want to approximate some analytically known function <math> f(x) </math> (whose integral is finite), as a sum of <math> K </math> Gaussians with different centers and widths. You could pretend that <math> f(x) </math> (or some scaling of it) was a probability distribution, draw <math> N </math> points from it and do the GMM thing to find the approximating Gaussians. Now take the limit <math> N\rightarrow \infty </math> , figure out how sums become integrals, and write down an iterative method for fitting Gaussians to a given <math> f(x) </math>. Does it work? (You can assume that well-defined definite integrals can be done numerically.) | ||

<b> Back To: </b> [[Eleisha Jackson]] | <b> Back To: </b> [[Eleisha Jackson]] |

## Latest revision as of 11:59, 3 April 2014

** To Calculate: **

1. Draw a sample of 100 points from the uniform distribution **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle U(0,1) }**
. This is your data set. Fit GMM models to your sample (now considered as being on the interval **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle -\infty < x < \infty }**
) with increasing numbers of components **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle K, }**
at least **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle K=1,\ldots,5 }**
. Plot your models. Do they get better as K increases? Did you try multiple starting values to find the best (hopefully globally best) solutions for each **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle K }**
?

2. Multiplying a lot of individual likelihoods will often underflow. (a) On average, how many values drawn from U(0,1) can you multiply before the product underflows to zero? (b) What, analytically, is the distribution of the sum of **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N }**
independent values **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \log(U) }**
, where **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle U\sim U(0,1 }**
)? (c) Is your answer to (a) consistent with your answer to (b)?

** To Think About: **

1. Suppose you want to approximate some analytically known function **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) }**
(whose integral is finite), as a sum of **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle K }**
Gaussians with different centers and widths. You could pretend that **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) }**
(or some scaling of it) was a probability distribution, draw **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N }**
points from it and do the GMM thing to find the approximating Gaussians. Now take the limit **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle N\rightarrow \infty }**
, figure out how sums become integrals, and write down an iterative method for fitting Gaussians to a given **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x) }**
. Does it work? (You can assume that well-defined definite integrals can be done numerically.)

** Back To: ** Eleisha Jackson