Seg17. The Multivariate Normal Distribution

From Computational Statistics (CSE383M and CS395T)
Jump to navigation Jump to search

Skilled problem

problem 1

Calculate the Jacobian determinant of the transformation of variables defined by

<math>y_1 = x_1/x_2, \qquad y_2 = x_2^2</math>

<math> J(x_1,x_2) = \begin{bmatrix}

\frac{dy_1}{dx_1} & \frac{dy_1}{dx_2} \\ \frac{dy_2}{dx_1} & \frac{dy_2}{dx_2} \\ \end{bmatrix} = \begin{bmatrix} \frac1{x_2} & -\frac{x_1}{x_2^2} \\ 0 & 2x_2 \\ \end{bmatrix},

|J(x_1,x_2)| = 2

</math>

problem 2

Consider the 3-dimensional multivariate normal over <math>(x_1,x_2,x_3)</math> with <math>\mu = (-1,-1,-1)</math> and

<math>\Sigma^{-1} = \left( \begin{array}{ccc}

5 & -1 & 2 \\
-1 & 8 & 1 \\
2 & 1 & 4

\end{array} \right)</math>. (Note the matrix inverse notation.)

What are 2-dimensional <math>\mu</math> and <math>\Sigma^{-1}</math> for

(a) the distribution on the slice <math>x_3=0</math>?

(b) the marginalization over <math>x_3</math>?

Hint: The answers are all simple rationals, but I had to use Mathematica to work them out.

The normal m.v.n for X is

<math> N(x|\mu,\Sigma) = \frac1{(2\pi)^\frac32 \cdot det(\Sigma)^\frac12} exp[-\frac12(x-\mu)^T\Sigma^{-1}(x-\mu)] </math>

(a)

From mathematica, we get

<math>\Sigma = \begin{bmatrix} \frac{31}{115} & \frac6{115} & -\frac{17}{115} \\ \frac6{115} & \frac{16}{115} & -\frac7{115} \\ -\frac{17}{115} & -\frac7{115} & \frac{39}{115} \\ \end{bmatrix} </math>

Let <math> Y_1 = \begin{bmatrix} x_1 \\ x_2 \\ \end{bmatrix}, Y_2 = x_3</math>

And we get <math>E(Y_1) = (-1,-1), E(Y_2) = -1 </math>

We get <math> cov(Y_1,Y_2) = \begin{bmatrix} \Sigma_{11} & \Sigma_{12} \\ \Sigma_{21} & \Sigma_{22} \\ \end{bmatrix} </math>

Where <math> \Sigma_{11} = \begin{bmatrix} \frac{31}{115} & \frac6{115} \\ \frac6{115} & \frac{16}{115} \\ \end{bmatrix},

\Sigma_{12} = \begin{bmatrix} -\frac{17}{115} \\ -\frac7{115} \\ \end{bmatrix},

\Sigma_{21} = \begin{bmatrix} -\frac{17}{115} & -\frac7{115} \\ \end{bmatrix},

\Sigma_{22} = \begin{bmatrix} \frac{39}{115} \\ \end{bmatrix} </math>

We get

<math> Cov[Y_1|Y_2=0] = \Sigma_{11} - \Sigma_{12} \cdot \Sigma_{22}^{-1} \cdot \Sigma_{21} = \begin{bmatrix} \frac{8}{39} & \frac1{39} \\ \frac1{39} & \frac5{39} \\ \end{bmatrix} </math>


<math>Mean[Y_1|Y_2=0] = E(Y_1) + \Sigma_{12}*\Sigma_{22}^{-1}*(0-E(Y_2)) = \begin{bmatrix} -1 \\ -1 \\ \end{bmatrix} + \Sigma_{12}*\Sigma_{22}^{-1} = \begin{bmatrix} -\frac{56}{39} \\ -\frac{46}{39} \\ \end{bmatrix},

</math>

(b)

The mean of the marginalization is simply (-1,-1).

The covariance matrix is all the interesting row and column,

<math> \begin{bmatrix} \frac{8}{39} & \frac1{39} \\ \frac1{39} & \frac5{39} \\ \end{bmatrix} </math>
--- The Conditionals you found for part a of this question. Where were those derived? Were they in the slides somewhere? - LoriL

--- Hi LoriL, the equations are shown in wikipedia, just google multivariate normal distribution, also Tameem has a detailed provement pasted on his webpage. - Jin 20:51, 11 April 2013 (CDT)

Thought problem