# Segment 21 Sanmit Narvekar

Jump to navigation Jump to search

## Segment 21

#### To Calculate

1. Consider a 2-dimensional multivariate normal distribution of the random variable $\displaystyle (b_1,b_2)$ with 2-vector mean $\displaystyle (\mu_1,\mu_2)$ and 2x2 matrix covariance $\displaystyle \Sigma$ . What is the distribution of $\displaystyle b_1$ given that $\displaystyle b_2$ has the particular value $\displaystyle b_c$ ? In particular, what is the mean and standard deviation of the conditional distribution of $\displaystyle b_1$ ? (Hint, either see Wikipedia "Multivariate normal distribution" for the general case, or else just work out this special case.)

According to WIkipedia, the conditional distribution of X_1 given X_2 is (note this is mean and variance, not standard deviation):

$\displaystyle X_1|X_2=x_2 \ \sim\ \mathcal{N}\left(\mu_1+\frac{\sigma_1}{\sigma_2}\rho( x_2 - \mu_2),\, (1-\rho^2)\sigma_1^2\right).$

where

$\displaystyle \rho_{X,Y}={\mathrm{cov}(X,Y) \over \sigma_X \sigma_Y}$

Thus, we can calculate the conditional mean and standard deviation as follows:

$\displaystyle \rho = \frac{\Sigma_{12}}{\sqrt{\Sigma_{11}\Sigma_{22}}}$

$\displaystyle \mu = \mu_1 + \frac{\sqrt{\Sigma_{11}}}{\sqrt{\Sigma_{22}}} \frac{\Sigma_{12}}{\sqrt{\Sigma_{11} \Sigma_{22}}} (b_c - \mu_2) = \mu_1 + \frac{\Sigma_{12}}{\Sigma_{22}} (b_c - \mu_2)$

$\displaystyle \sigma = \sqrt{ \left( 1 - \frac{\Sigma_{12}^2}{\Sigma_{11}\Sigma_{22}} \right) \Sigma_{11} } = \sqrt{ \frac{\Sigma_{22}\Sigma_{11} - \Sigma_{12}^2}{\Sigma_{22}}}$

2. Same, but marginalize over $\displaystyle b_2$ instead of conditioning on it.

To marginalize over b2, we simply remove those rows/columns from the covariance matrix, and also from the mean vector. Thus, the marginalized mean and standard deviation are:

$\displaystyle \mu = \mu_1$

$\displaystyle \sigma = \sqrt{\Sigma_{11}}$

#### To Think About

1. Why should it be called the Fisher Information Matrix? What does it have to do with "information"?

2. Go read (e.g., in Wikipedia or elsewhere) about the "Cramer-Rao bound" and be prepared to explain what it is, and what it has to do with the Fisher Information Matrix.