Segment 16. Multiple Hypotheses

From Computational Statistics Course Wiki
Jump to: navigation, search

Watch this segment

(Don't worry, what you see statically below is not the beginning of the segment. Press the play button to start at the beginning.)

The direct YouTube link is

Links to the slides: PDF file or PowerPoint file


To Calculate

1. Simulate the following: You have M=50 p-values, none actually causal, so that they are drawn from a uniform distribution. Not knowing this sad fact, you apply the Benjamini-Hochberg prescription with Failed to parse (unknown error): \alpha=0.05 and possibly call some discoveries as true. By repeated simulation, estimate the probability of thus getting N wrongly-called discoveries, for N=0, 1, 2, and 3.

2. Does the distribution that you found in problem 1 depend on M? On Failed to parse (unknown error): \alpha ? Derive its form analytically for the usual case of Failed to parse (unknown error): \alpha \ll 1 ?

To Think About

1. Suppose you have M independent trials of an experiment, each of which yields an independent p-value. Fisher proposed combining them by forming the statistic

Failed to parse (unknown error): S = -2\sum_{i=0}^{i=M}\log(p_i)

Show that, under the null hypothesis, S is distributed as Failed to parse (unknown error): \text{Chisquare}(2M) and describe how you would obtain a combined p-value for this statistic.

2. Fisher is sometimes credited, on the basis of problem 1, with having invented "meta-analysis", whereby results from multiple investigations can be combined to get an overall more significant result. Can you see any pitfalls in this?

Class Activity

P-value follow-ups

Here is John's written up solution: Pvalue Examples Solutions.