# Difference between revisions of "Eleisha's Segment 20: Non-linear Least Squares Fitting"

(3 intermediate revisions by the same user not shown) | |||

Line 3: | Line 3: | ||

<b> To Calculate </b> | <b> To Calculate </b> | ||

− | 1. (See lecture slide 3.) For one-dimensional x, the model y(x | \mathbf b) is called "linear" if <math> y(x | \mathbf b) =\sum_k b_k X_k(x) </math> , where <math> X_k(x) </math> are arbitrary known functions of <math> x </math>. Show that minimizing <math> \chi^2</math> produces a set of linear equations (called the "normal equations") for the parameters <math>b_k </math>. | + | 1. (See lecture slide 3.) For one-dimensional x, the model <math> y(x | \mathbf b) </math> is called "linear" if <math> y(x | \mathbf b) =\sum_k b_k X_k(x) </math> , where <math> X_k(x) </math> are arbitrary known functions of <math> x</math>. Show that minimizing <math> \chi^2</math> produces a set of linear equations (called the "normal equations") for the parameters <math>b_k </math>. |

2. A simple example of a linear model is <math> y(x | \mathbf b) = b_0 + b_1 x </math> , which corresponds to fitting a straight line to data. What are the MLE estimates of <math> b_0</math> and <math>b_1</math> in terms of the data: <math>x_i's, y_i's </math>, and <math> \sigma_i's</math>? | 2. A simple example of a linear model is <math> y(x | \mathbf b) = b_0 + b_1 x </math> , which corresponds to fitting a straight line to data. What are the MLE estimates of <math> b_0</math> and <math>b_1</math> in terms of the data: <math>x_i's, y_i's </math>, and <math> \sigma_i's</math>? |

## Latest revision as of 10:54, 12 April 2014

** To Calculate **

1. (See lecture slide 3.) For one-dimensional x, the model **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y(x | \mathbf b) }**
is called "linear" if **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y(x | \mathbf b) =\sum_k b_k X_k(x) }**
, where **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X_k(x) }**
are arbitrary known functions of **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x}**
. Show that minimizing **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi^2}**
produces a set of linear equations (called the "normal equations") for the parameters **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_k }**
.

2. A simple example of a linear model is **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle y(x | \mathbf b) = b_0 + b_1 x }**
, which corresponds to fitting a straight line to data. What are the MLE estimates of **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_0}**
and **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle b_1}**
in terms of the data: **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_i's, y_i's }**
, and **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sigma_i's}**
?

** To Think About **

1. We often rather casually assume a uniform prior **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\mathbf b)= \text{constant} }**
on the parameters **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf b }**
. If the prior is not uniform, then is minimizing **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \chi^2 }**
the right thing to do? If not, then what should you do instead? Can you think of a situation where the difference would be important?

2. What if, in lecture slide 2, the measurement errors were **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle e_i \sim \text{Cauchy}(0,\sigma_i) }**
instead of **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle e_i \sim N(0,\sigma_i) }**
? How would you find MLE estimates for the parameters **Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbf b }**
?

** Back To: ** Eleisha Jackson