Segment 21: Marginalize or Condition Uninteresting Fitted Parameters - 3/27/2013

Problem 1

1. Consider a 2-dimensional multivariate normal distribution of the random variable $(b_1,b_2)$ with 2-vector mean $(\mu_1,\mu_2)$ and 2x2 matrix covariance $\Sigma$. What is the distribution of $b_1$ given that $b_2$ has the particular value $b_c$? In particular, what is the mean and standard deviation of the conditional distribution of $b_1$? (Hint, either see Wikipedia "Multivariate normal distribution" for the general case, or else just work out this special case.)

Since this is along the lines of slicing, we can set our Mean and Covariance Matrix as follows

Mean = $E[b_1] + \Sigma_{12} \Sigma_{22}^{-1}{(b_c-E[b_2])}$

$\Sigma = \Sigma_{11} -\Sigma_{12} \Sigma_{22}^{-1} \Sigma_{21}$

Problem 2

2. Same, but marginalize over $b_2$ instead of conditioning on it.

$\Sigma=\Sigma_{11}, Mean=\mu_1$

To Think About 1

Why should it be called the Fisher Information Matrix? What does it have to do with "information"?
The Fisher Information Matrix has components that can tell us about the covariance matrix. We can use that information to see if particular random variables are independent to one another, or the variances of each particular random variable with a little bit of tweaking with the information.