Segment 20. Nonlinear Least Squares Fitting

From Computational Statistics Course Wiki
Jump to: navigation, search

Watch this segment

(Don't worry, what you see statically below is not the beginning of the segment. Press the play button to start at the beginning.)

The direct YouTube link is http://youtu.be/xtBCGPHRcb0

Links to the slides: PDF file or PowerPoint file

Problems

To Calculate

1. (See lecture slide 3.) For one-dimensional Failed to parse (unknown error): x , the model Failed to parse (unknown error): y(x | \mathbf b) is called "linear" if Failed to parse (unknown error): y(x | \mathbf b) = \sum_k b_k X_k(x) , where Failed to parse (unknown error): X_k(x) are arbitrary known functions of Failed to parse (unknown error): x . Show that minimizing Failed to parse (unknown error): \chi^2 produces a set of linear equations (called the "normal equations") for the parameters Failed to parse (unknown error): b_k .

2. A simple example of a linear model is Failed to parse (unknown error): y(x | \mathbf b) = b_0 + b_1 x , which corresponds to fitting a straight line to data. What are the MLE estimates of Failed to parse (unknown error): b_0 and Failed to parse (unknown error): b_1 in terms of the data: Failed to parse (unknown error): x_i 's, Failed to parse (unknown error): y_i 's, and Failed to parse (unknown error): \sigma_i 's?

To Think About

1. We often rather casually assume a uniform prior Failed to parse (unknown error): P(\mathbf b)= \text{constant} on the parameters Failed to parse (unknown error): \mathbf b . If the prior is not uniform, then is minimizing Failed to parse (unknown error): \chi^2 the right thing to do? If not, then what should you do instead? Can you think of a situation where the difference would be important?

2. What if, in lecture slide 2, the measurement errors were Failed to parse (unknown error): e_i \sim \text{Cauchy}(0,\sigma_i) instead of Failed to parse (unknown error): e_i \sim N(0,\sigma_i) ? How would you find MLE estimates for the parameters Failed to parse (unknown error): \mathbf b ?

Class Activity

Here is some data: Media:Chisqfitdata.txt

In class we will work on fitting this to some models as explained here.

Here are Bill's numerical answers, so that you can see whether you are on the right track (or whether Bill got it wrong!): Media:Chisqfitanswers.txt