Segment 16. Multiple Hypotheses
Watch this segment
(Don't worry, what you see statically below is not the beginning of the segment. Press the play button to start at the beginning.)
{{#widget:Iframe |url=http://www.youtube.com/v/w6AjduOEN2k&hd=1 |width=800 |height=625 |border=0 }}
The direct YouTube link is http://youtu.be/w6AjduOEN2k
Links to the slides: PDF file or PowerPoint file
Problems
To Calculate
1. Simulate the following: You have M=50 p-values, none actually causal, so that they are drawn from a uniform distribution. Not knowing this sad fact, you apply the Benjamini-Hochberg prescription with Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha=0.05} and possibly call some discoveries as true. By repeated simulation, estimate the probability of thus getting N wrongly-called discoveries, for N=0, 1, 2, and 3.
2. Does the distribution that you found in problem 1 depend on M? On Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha} ? Derive its form analytically for the usual case of Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \alpha \ll 1} ?
To Think About
1. Suppose you have M independent trials of an experiment, each of which yields an independent p-value. Fisher proposed combining them by forming the statistic
Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S = -2\sum_{i=0}^{i=M}\log(p_i)}
Show that, under the null hypothesis, S is distributed as Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \text{Chisquare}(2M)} and describe how you would obtain a combined p-value for this statistic.
2. Fisher is sometimes credited, on the basis of problem 1, with having invented "meta-analysis", whereby results from multiple investigations can be combined to get an overall more significant result. Can you see any pitfalls in this?
Class Activity
- Team 1 - Feb 21 Activity
- [Team Girls + Sanmit - Feb 21 Activity]
- Team3-021714part2
- Feb20-Team4-P-value follow up
Here is John's written up solution: Pvalue Examples Solutions.
