Segment 16. Multiple Hypotheses

From Computational Statistics Course Wiki
Jump to navigation Jump to search

Watch this segment

(Don't worry, what you see statically below is not the beginning of the segment. Press the play button to start at the beginning.)

{{#widget:Iframe |url=http://www.youtube.com/v/w6AjduOEN2k&hd=1 |width=800 |height=625 |border=0 }}

The direct YouTube link is http://youtu.be/w6AjduOEN2k

Links to the slides: PDF file or PowerPoint file

Problems

To Calculate

1. Simulate the following: You have M=50 p-values, none actually causal, so that they are drawn from a uniform distribution. Not knowing this sad fact, you apply the Benjamini-Hochberg prescription with and possibly call some discoveries as true. By repeated simulation, estimate the probability of thus getting N wrongly-called discoveries, for N=0, 1, 2, and 3.

2. Does the distribution that you found in problem 1 depend on M? On ? Derive its form analytically for the usual case of ?

To Think About

1. Suppose you have M independent trials of an experiment, each of which yields an independent p-value. Fisher proposed combining them by forming the statistic

Show that, under the null hypothesis, S is distributed as and describe how you would obtain a combined p-value for this statistic.

2. Fisher is sometimes credited, on the basis of problem 1, with having invented "meta-analysis", whereby results from multiple investigations can be combined to get an overall more significant result. Can you see any pitfalls in this?

Class Activity

P-value follow-ups

Here is John's written up solution: Pvalue Examples Solutions.