27

From Computational Statistics Course Wiki
Jump to navigation Jump to search

1. Write down an expression for the probability of the file's data given some values for the parameters Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta} and c.

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\beta, c|data) = \frac{P(data|\beta,c)P(\beta,c)}{\int_0^1 \int_0^{\infty}P(data|\beta,c)P(\beta,c)} dc d\beta }

Let us assume a uniform prior on c and Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta>} . More specifically, although Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta \epsilon (0, \infty) } , it is most likely no greater than 10,000, so we can place a uniform prior on Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta>} in the range 0 to 10,000. Based on this uniform prior:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\beta, c|data) \propto P(data|\beta, c)}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\beta, C|data) = \prod_{i} [P_{1i}C + P_{0i}(1-c)]P(\beta, c)}

Where

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P_{1i} = \frac{1}{\beta}e^{-\frac{x}{\beta}}}

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P_{0i} = \frac{2}{\pi(1+x^2)}}

2. Calculate numerically the maximum likelihood values of \beta and c.

plugging into fmin in python:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \beta = 1.503 }

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle c = .35956 }

Code below:

import scipy as sc
import scipy.optimize as opt
import numpy as np
import math
samples = np.loadtxt('Mixtureevals.txt')
def prob_1(data, beta):
   return (1./beta)*np.exp(-data/beta)
   
def prob_2(data):
   return (2./sc.pi)/(1+np.power(data,2))
def likeHood(data, beta, c):
   probC = np.zeros(len(data))
   probC = np.log(prob_1(data,beta)*c + prob_2(data)*(1.-c))
   return np.sum(probC)

def nl (params, data):
   beta, c = params
   return -likeHood(data, beta, c)
x0 = [1, 1]
beta, c = opt.fmin(nl, x0 , args=(samples,))
print beta
print c

3. Estimate numerically the Bayes posterior distribution of \beta, marginalizing over c as a nuisance parameter. (You'll of course have to make some assumption about priors.)

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\beta|data) = \int_0^1 P(\beta|data,c)P(c) dc }


Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle P(\beta|data,c) = \frac{P(C|data,\beta)*P(B)}{\int_0^{\infty} P(C|\beta, data)P(B)db} }

A uniform prior on P(c) is assumed.