Travis: Segment 20

From Computational Statistics (CSE383M and CS395T)
Jump to navigation Jump to search

To Calculate

1. (See lecture slide 3.) For one-dimensional <math>x</math>, the model <math>y(x | \mathbf b)</math> is called "linear" if <math>y(x | \mathbf b) = \sum_k b_k X_k(x)</math>, where <math>X_k(x)</math> are arbitrary known functions of <math>x</math>. Show that minimizing <math>\chi^2</math> produces a set of linear equations (called the "normal equations") for the parameters <math>b_k</math>.

2. A simple example of a linear model is <math>y(x | \mathbf b) = b_0 + b_1 x</math>, which corresponds to fitting a straight line to data. What are the MLE estimates of <math>b_0</math> and <math>b_1</math> in terms of the data: <math>x_i</math>'s, <math>y_i</math>'s, and <math>\sigma_i</math>'s?

To Think About

1. We often rather casually assume a uniform prior <math>P(\mathbf b)= \text{constant}</math> on the parameters <math>\mathbf b</math>. If the prior is not uniform, then is minimizing <math>\chi^2</math> the right thing to do? If not, then what should you do instead? Can you think of a situation where the difference would be important?

2. What if, in lecture slide 2, the measurement errors were <math>e_i \sim \text{Cauchy}(0,\sigma_i)</math> instead of <math>e_i \sim N(0,\sigma_i)</math>? How would you find MLE estimates for the parameters <math>\mathbf b</math>?