Difference between revisions of "Eleisha's Segment 20: Non-linear Least Squares Fitting"

From Computational Statistics Course Wiki
Jump to navigation Jump to search
(Created page with " <b> To Calculate </b> 1. (See lecture slide 3.) For one-dimensional x, the model y(x | \mathbf b) is called "linear" if <math> y(x | \mathbf b) =\sum_k b_k X_k(x) </math> ,...")
 
Line 3: Line 3:
 
<b> To Calculate </b>
 
<b> To Calculate </b>
  
1. (See lecture slide 3.) For one-dimensional x, the model y(x | \mathbf b) is called "linear" if <math> y(x | \mathbf b) =\sum_k b_k X_k(x) </math> , where <math> X_k(x) </math> are arbitrary known functions of x. Show that minimizing <math> \chi^2</math> produces a set of linear equations (called the "normal equations") for the parameters <math>b_k </math>.
+
1. (See lecture slide 3.) For one-dimensional x, the model y(x | \mathbf b) is called "linear" if <math> y(x | \mathbf b) =\sum_k b_k X_k(x) </math> , where <math> X_k(x) </math> are arbitrary known functions of <math> x </math>. Show that minimizing <math> \chi^2</math> produces a set of linear equations (called the "normal equations") for the parameters <math>b_k </math>.
  
 
2. A simple example of a linear model is <math> y(x | \mathbf b) = b_0 + b_1 x </math> , which corresponds to fitting a straight line to data. What are the MLE estimates of <math> b_0</math> and <math>b_1</math> in terms of the data: <math>x_i's, y_i's </math>, and <math> \sigma_i's</math>?
 
2. A simple example of a linear model is <math> y(x | \mathbf b) = b_0 + b_1 x </math> , which corresponds to fitting a straight line to data. What are the MLE estimates of <math> b_0</math> and <math>b_1</math> in terms of the data: <math>x_i's, y_i's </math>, and <math> \sigma_i's</math>?

Revision as of 11:19, 3 April 2014


To Calculate

1. (See lecture slide 3.) For one-dimensional x, the model y(x | \mathbf b) is called "linear" if Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y(x | \mathbf b) =\sum_k b_k X_k(x) } , where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X_k(x) } are arbitrary known functions of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x } . Show that minimizing Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi^2} produces a set of linear equations (called the "normal equations") for the parameters Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_k } .

2. A simple example of a linear model is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y(x | \mathbf b) = b_0 + b_1 x } , which corresponds to fitting a straight line to data. What are the MLE estimates of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_0} and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_1} in terms of the data: Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_i's, y_i's } , and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sigma_i's} ?

To Think About

1. We often rather casually assume a uniform prior Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\mathbf b)= \text{constant} } on the parameters Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf b } . If the prior is not uniform, then is minimizing Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi^2 } the right thing to do? If not, then what should you do instead? Can you think of a situation where the difference would be important?

2. What if, in lecture slide 2, the measurement errors were Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle e_i \sim \text{Cauchy}(0,\sigma_i) } instead of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle e_i \sim N(0,\sigma_i) } ? How would you find MLE estimates for the parameters Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf b }  ?

Back To: Eleisha Jackson