Segment 17: The Multivariate Normal Distribution - 3/8/2013

Problem 1

Calculate the Jacobian determinant of the transformation

$y_1 = \frac{x_1}{x_2}$ $y_2 = x_2^2$
$\left[\begin{array}{ccc}\frac{dy_1}{dx_1} & \frac{dy_1}{dx_2} \\ \frac{dy_2}{dx_1} & \frac{dy_2}{dx_1} \end{array} \right]$ = $\left[\begin{array}{ccc}\frac{1}{x_2} & \frac{-x_1}{x_2^2} \\ 0 & 2x_2^2 \end{array}\right ]$

Problem 1

Consider the 3-dimensional multivariate normal over (x1,x2,x3)
What are 2-dimensional μ and Σ − 1 for
(a) the distribution on the slice x3 = 0?
(b) the marginalization over x3?

Part(a)
Let $Y_1 = \left[\begin{array}{ccc}x_1\\ x_2\end{array}\right]$ and $Y_2 = [x_3]$
Their corresponding mean matrix is matching up their respective means. Mean of $Y_1 = \left[\begin{array}{ccc}-1\\ -1\end{array}\right]$ and Mean of $Y_2 = [-1]$
Since we'll need the covariance matrix, using my calculator my covariance matrix for x1,x2, x3 = $\left[\begin{array}{ccc}\frac{31}{115} & \frac{6}{115} & \frac{-17}{115}\\ \frac{6}{115} & \frac{16}{115} & \frac{-7}{115}\\ \frac{-17}{115} & \frac{-7}{115} & \frac{39}{115}\end{array}\right]$
To break this down to make the covariance matrix for Y1 and Y2 we have: $\left[\begin{array}{ccc}Cov(Y_1, Y_1)& Cov(Y_1, Y_2)\\ Cov(Y_2, Y_1) & Cov(Y_2, Y_2)\end{array}\right]$ = $\left[\begin{array}{ccc}Var(Y_1)& Cov(Y_1, Y_2)\\ Cov(Y_2, Y_1) & Var(Y_2)\end{array}\right]$
From observation, to fill in the covariances, we can read the covariances as Cov(Row, Column) where we read in what row and column we want from the corresponding variables represented in Y1 and Y2 from the covariance matrix. Thus, our covariance matrix is as follows:
$\left[\begin{array}{ccc}\sum_{11}& \sum_{12}\\ \sum_{21} & \sum_{22}\end{array}\right] where \sum_{11}$ = $\left[\begin{array}{ccc}\frac{31}{115} & \frac{6}{115}\\ \frac{6}{115} & \frac{16}{115}\end{array}\right], \sum_{12}$ = $\left[\begin{array}{ccc}\frac{-17}{115}\\ \frac{-7}{115}\end{array}\right], \sum_{21}$ = $\left[\begin{array}{ccc}\frac{-17}{115} & \frac{-7}{115}\end{array}\right]$ and $\sum_{22} = [\frac{39}{115}$]

Thanks to Tameem's reference, I was able to get the equations to represent this question. I am still unsure how to actually derive these equations, but the following equations are needed to find the distribution over the slice x3 = 0

$Mean[Y_1| Y_2=0] = E(Y_1) + \Sigma_{12}*\Sigma_{22}^{-1}*(0-E(Y_2)) = \begin{bmatrix} -1 \\ -1 \\ \end{bmatrix} + \Sigma_{12}*\Sigma_{22}^{-1} = \begin{bmatrix} -\frac{56}{39} \\ -\frac{46}{39} \\ \end{bmatrix}$
$Cov[Y_1|Y_2=0] = \Sigma_{11} - \Sigma_{12} \cdot \Sigma_{22}^{-1} \cdot \Sigma_{21} = \begin{bmatrix} \frac{8}{39} & \frac1{39} \\ \frac1{39} & \frac5{39} \\ \end{bmatrix}$
${\Sigma}^{-1} = \begin{bmatrix} 5 & -1 \\[0.4em] -1 & 8 \end{bmatrix}$

Part (b)
Since we are marginalizing over x3, all we do is take the important columns that we want for our distribution from the original mean vector and covariance vector. Thus:
$Mean = \begin{bmatrix} -1 \\ -1 \end{bmatrix}$
$Covariance = \begin{bmatrix} \frac{31}{115} & \frac{6}{115} \\ \frac{6}{115} & \frac{16}{115} \\ \end{bmatrix}$
$\begin{bmatrix} {4} & {-\frac{3}{2}} \\[0.4em] {-\frac{3}{2}} & {\frac{31}{4}} \end{bmatrix}$