Seg17. The Multivariate Normal Distribution

Skilled problem

problem 1

Calculate the Jacobian determinant of the transformation of variables defined by

$y_1 = x_1/x_2, \qquad y_2 = x_2^2$

$J(x_1,x_2) = \begin{bmatrix} \frac{dy_1}{dx_1} & \frac{dy_1}{dx_2} \\ \frac{dy_2}{dx_1} & \frac{dy_2}{dx_2} \\ \end{bmatrix} = \begin{bmatrix} \frac1{x_2} & -\frac{x_1}{x_2^2} \\ 0 & 2x_2 \\ \end{bmatrix}, |J(x_1,x_2)| = 2$

problem 2

Consider the 3-dimensional multivariate normal over $(x_1,x_2,x_3)$ with $\mu = (-1,-1,-1)$ and

$\Sigma^{-1} = \left( \begin{array}{ccc} 5 & -1 & 2 \\ -1 & 8 & 1 \\ 2 & 1 & 4  \end{array} \right)$. (Note the matrix inverse notation.)

What are 2-dimensional $\mu$ and $\Sigma^{-1}$ for

(a) the distribution on the slice $x_3=0$?

(b) the marginalization over $x_3$?

Hint: The answers are all simple rationals, but I had to use Mathematica to work them out.

The normal m.v.n for X is

$N(x|\mu,\Sigma) = \frac1{(2\pi)^\frac32 \cdot det(\Sigma)^\frac12} exp[-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)]$

(a)

From mathematica, we get

$\Sigma = \begin{bmatrix} \frac{31}{115} & \frac6{115} & -\frac{17}{115} \\ \frac6{115} & \frac{16}{115} & -\frac7{115} \\ -\frac{17}{115} & -\frac7{115} & \frac{39}{115} \\ \end{bmatrix}$

Let $Y_1 = \begin{bmatrix} x_1 \\ x_2 \\ \end{bmatrix}, Y_2 = x_3$

And we get $E(Y_1) = (-1,-1), E(Y_2) = -1$

We get $cov(Y_1,Y_2) = \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \\ \end{bmatrix}$

Where $\Sigma_{11} = \begin{bmatrix} \frac{31}{115} & \frac6{115} \\ \frac6{115} & \frac{16}{115} \\ \end{bmatrix}, \Sigma_{12} = \begin{bmatrix} -\frac{17}{115} \\ -\frac7{115} \\ \end{bmatrix}, \Sigma_{21} = \begin{bmatrix} -\frac{17}{115} & -\frac7{115} \\ \end{bmatrix}, \Sigma_{22} = \begin{bmatrix} \frac{39}{115} \\ \end{bmatrix}$

We get

$Cov[Y_1|Y_2=0] = \Sigma_{11} - \Sigma_{12} \cdot \Sigma_{22}^{-1} \cdot \Sigma_{21} = \begin{bmatrix} \frac{8}{39} & \frac1{39} \\ \frac1{39} & \frac5{39} \\ \end{bmatrix}$

$Mean[Y_1|Y_2=0] = E(Y_1) + \Sigma_{12}*\Sigma_{22}^{-1}*(0-E(Y_2)) = \begin{bmatrix} -1 \\ -1 \\ \end{bmatrix} + \Sigma_{12}*\Sigma_{22}^{-1} = \begin{bmatrix} -\frac{56}{39} \\ -\frac{46}{39} \\ \end{bmatrix},$

(b)

The mean of the marginalization is simply (-1,-1).

The covariance matrix is all the interesting row and column,

$\begin{bmatrix} \frac{8}{39} & \frac1{39} \\ \frac1{39} & \frac5{39} \\ \end{bmatrix}$
--- The Conditionals you found for part a of this question. Where were those derived? Were they in the slides somewhere? - LoriL

--- Hi LoriL, the equations are shown in wikipedia, just google multivariate normal distribution, also Tameem has a detailed provement pasted on his webpage. - Jin 20:51, 11 April 2013 (CDT)