Segment 12: Segment 12. P-Value Tests - 2/22/2012

From Computational Statistics (CSE383M and CS395T)
Jump to navigation Jump to search

Problem 1

What is the critical region for a 5% two-sided test if, under the null hypothesis, the test statistic is distributed as Student(0,σ,4)? That is, what values of the test statistic disprove the null hypothesis with p < 0.05? (OK to use Python, MATLAB, or Mathematica.)
Critical region is X < -2.776 or X > 2.776

Problem 2

For an exponentially distributed test statistic with mean μ (under the null hypothesis), when is the the null hypothesis disproved with p < 0.01 for a one-sided test? for a two-sided test?
We know that the lambda = 1/mean
For one-sided:
<math>\int_t^{\infty} \lambda e^{-\lambda x}dx = -e^{-\lambda x}]_t^{\infty} </math>
<math> e^{-\frac{1}{\mu}t} = 0.01 \longrightarrow t = -\mu ln(0.01)</math>
Thus <math> X > -\mu ln(0.01)</math>
For two-sided:
<math>\int_t^{\infty} \lambda e^{-\lambda x}dx = -e^{-\lambda x}]_t^{\infty} </math>
<math> e^{-\frac{1}{\mu}t} = 0.005 \longrightarrow t = -\mu ln(0.005)</math>
<math>\int_0^{t} \lambda e^{-\lambda x}dx = -e^{-\lambda x}]_t^{\infty} </math>
<math> - e^{-\frac{1}{\mu}t} + 1 = 0.005 \longrightarrow - e^{-\frac{1}{\mu}t} = -0.995 \longrightarrow t = -\mu ln(0.995)</math>
Thus <math> X > -\mu ln(0.005)</math> or <math> 0 \leq X \leq -\mu ln(0.995)</math>

To Think About 1

P-value tests require an initial choice of a test statistic. What goes wrong if you choose a poor test statistic? What would make it poor?
If you choose a poor test statistic you'll get many results that are neither extreme or similar to the mean, such that you will get many results that will fail to reject the null hypothesis at values of Z = 1.4 or Z = -1.2. This may be caused that there may be too many confounding variables in your problem, or not enough data is pulled to be a good test statistic.

To Think About 2

If the null hypothesis is that a coin is fair, and you record the results of N flips, what is a good test statistic? Are there any other possible test statistics?
A good test statistic would be something that is extreme or close to a Z value of zero. Extremes show that something interesting is going on in the data so you do more research into the problem. Any Z value close to zero shows that the data indeed is close to the expected mean predicted.

To Think About 3

Why is it so hard for a Bayesian to do something as simple as, given some data, disproving a null hypothesis? Can't she just compute a Bayes odds ratio, P(null hypothesis is true)/P(null hypothesis is false) and derive a probability that the null hypothesis is true?
Because in doing experiments, nothing is ever true. We can only disprove that it is not the correct mean or statistic we are trying to find.

Class Activity

Worked with Noah and Sean Trettel