Dan's Segment 30

From Computational Statistics (CSE383M and CS395T)
Jump to navigation Jump to search

To Calculate

1. In the slides we were using a concave function, but <math>x^2</math> is convex, so we reverse the inequality and have <math>function(interpolation) \le interpolation(function)</math>

In this case the function is <math>x^2</math> and the interpolation is the mean, so the specific inequality here is:

<math> (\sum_{i=1}^{N} \frac{x_i}{N})^2 \le \sum_{i=1}^{N} \frac{{x_i}^2}{N} </math>

And the problem statement is satisfied.

2. This problem is best approached by looking at the case where g(xm) = f(xm) as the case where g > f is self-explanatory from the problem statement. If f(xm) is at a local maximum, its derivative is 0. If g = f, and its derivative is not 0, it would cross over the function f and thus not be greater than or equal to f at all values of x. Thus, the derivative of g must be zero and therefor it is at a local maximum.

I do have a question about this though. What is to stop the function g from being at a local minimum at that particular x value or even just at some sort of inflection point? If the function always had to be concave for all values of x that would not be a problem, but for the general case I'm not sure this holds.

To Think About

1. I'm pretty sure it is being applied to a mixture of gaussian functions, or at least to some mixture of functions with that general form.

2. It allows us to use that known function in order to make sure we are always increasing the value of the unknown function. Basically, this is what makes EM methods possible. We always want to increase the likelihood function, but we do not know explicitly what it is. However, if we can iteratively created a function we know is less than the likelihood function and find its maximum, we know we are increasing likelihood.