Segment 12...P-Value Tests

From Computational Statistics Course Wiki
Jump to navigation Jump to search

Problems from Segment 12. P-Value Tests

To Calculate

1. What is the critical region for a 5% two-sided test if, under the null hypothesis, the test statistic is distributed as ? That is, what values of the test statistic disprove the null hypothesis with p < 0.05? (OK to use Python, MATLAB, or Mathematica.)

I am guessing this means since we can't have , even though in Segment 8. Some Standard Distributions the format is .

So we want the value where the cdf which in MATLAB looks like:


So we get which means a test value T such that disproves the null hypothesis with .

2. For an exponentially distributed test statistic with mean (under the null hypothesis), when is the the null hypothesis disproved with p < 0.01 for a one-sided test? for a two-sided test?

First let's get the cdf of the exponential distribution with mean , which implies the parameter is - we integrate the probability distribution w.r.t to t from 0 to x (start at 0 because p(t)=0 for negative values).

So the CDF is . Let's invert this:

So for one sided with  :
If close to is considered extreme, put in . Then values less than this disprove the null hypothesis.
If large values of are considered extreme, put in . Then values greater than this disprove the null hypothesis.

For two sided with  :
Use and . Then values such that and disprove the null hypothesis.

To Think About
1. P-value tests require an initial choice of a test statistic. What goes wrong if you choose a poor test statistic? What would make it poor?

2. If the null hypothesis is that a coin is fair, and you record the results of N flips, what is a good test statistic? Are there any other possible test statistics?

3. Why is it so hard for a Bayesian to do something as simple as, given some data, disproving a null hypothesis? Can't she just compute a Bayes odds ratio, P(null hypothesis is true)/P(null hypothesis is false) and derive a probability that the null hypothesis is true?

Class Activity

P-Value Extravaganza

1. Its a function(al) , where is the set of all possible states that can take.
2. An event is some set of possible outcomes.
5. A large p value does not tell us as much as a small value. The null hypothesis is false, but we don't have enough evidence to support that claim. A p value never supports a hypothesis, it can only reject it.
7. can't get website to load!!!
8. What is the difference between two random variables? If you have two PDFs, you don't subtract them from each other, you draw from each and compute a new distribution from their difference.
9. Proof uses the characteristic functions.
10. 11. mean = , since we have tries with probability.
variance = , we use indicator functions to show. 12. P-values are the fraction of the area under the curve, so just like the cdf smooths out a bumpy distribution, because the p-values come from the area under the curve they are normally distributed.

What was wrong with the video?
1. .4575>.5 WHAT
2. standard deviation is 1.87 when the statistic they are measuring ranges from 0 to 1.
3. is the MLE but the null hypothesis is that that we are trying to disprove that is the case!

Back to Segment 12. P-Value Tests or Ellen Le