# Eleisha's Segment 14: Bayesian Criticism of P-Values

To Calculate:

1. Suppose the stopping rule is "flip exactly 10 times" and the data is that 8 out of 10 flips are heads. With what p-value can you rule out the hypothesis that the coin is fair? Is this statistically significant?

$\displaystyle \text{p - value} = \sum_{k = 8}^{10} {10\choose k} (0.5)^k(1 - p)^{10 - k}$

For a one - sided test the p-value is: 0.0546875

For a two - sided test the p-value is: 0.109375

Here is the python code that was used to calculate the p-values:

import math, scipy.stats
import numpy as np

def get_p_value(k, n, p):
p_value  = 0.0
for i in xrange(k, n + 1):
#print i
binom = scipy.stats.binom.pmf(i,n,p)
p_value = p_value + binom
return p_value

print "Stop Rule: Flip 10 times, 8 out of 10 are heads"
print "P-value One-Sided = " + str(str(get_p_value(8, 10, 0.5)))
print "P-value Two Sided = " + str(2*get_p_value(8, 10, 0.5))


The output produced by the code above is:

Stop Rule: Flip 10 times, 8 out of 10 are heads
P-value One-Sided = 0.0546875
P-value Two Sided = 0.109375


This is not statistically significant. With a requirement that p < 0.05, this you can not rule out the hypothesis that the coin is fair.

2. Suppose that, as a Bayesian, you see 10 flips of which 8 are heads. Also suppose that your prior for the coin being fair is 0.75. What is the posterior probability that the coin is fair? (Make any other reasonable assumptions about your prior as necessary.)

$\displaystyle H_A = \text{The coin is fair}$

$\displaystyle H_B = \text{The coin is unfair}$

$\displaystyle P(H_A) = 0.75$

$\displaystyle P(H_B) = 0.25$

$\displaystyle P(H_A | data) = P(data | (H_A)P(H_A)= { 10 \choose 8}*(0.75)*(0.5)^8*(0.5)^2 = 0.032959$

$\displaystyle P(H_B | data) = P(data | (H_B)P(H_B)= { 10 \choose 8}*(0.25)*\int_0^1p^8(1- p)^2 = 0.227273$

If you normalize by: $\displaystyle P(H_A | data) + P(H_B | data)$

$\displaystyle P(H_A | data) = 0.591869$

$\displaystyle P(H_B | data) = 0.408131$

So the posterior probability that the coin is fair is: 0.591869

Below is the Mathematica code that was used to calculate the probabilities:

3. For the experiment in the segment, what if the stopping rule was (perversely) "flip until I see five consecutive heads followed immediately by a tail, then count the total number of heads"? What would be the p-value?