Judicious Use of Multiple Hypothesis Tests
When analyzing a table of statistical results, one must first decide whether adjustment of significance levels is appropriate. If the main goal is hypothesis generation or initial screening for potential conservation problems, then it may be appropriate to use the standard comparisonwise significance level to avoid Type II errors (not detecting real differences or trends). If the main goal is rigorous testing of a hypothesis, however, then an adjustment for multiple tests is needed. To control the familywise Type I error rate (the probability of rejecting at least one true null hypothesis), sequential modifications of the standard Bonferroni method, such as Holm's method, will provide more statistical power than the standard Bonferroni method. Additional power may be achieved through procedures that control the false discovery rate (FDR) (the expected proportion of false positives among tests found to be significant). Holm's sequential Bonferroni method and two FDR-controlling procedures were applied to the results of multiple-regression analyses of the relationship between habitat variables and the abundance of 25 species of forest birds in Japan, and the FDR-controlling procedures provided considerably greater statistical power.
Keywords: Bonferroni method; Holm's method; descubrimiento falso; false discovery rate; generación de hipótesis; hypothesis generation; multiple hypothesis testing; método Bonferroni; método de Holm; prueba de hipótesis múltiples; q value; tasa de valor q
Document Type: Research Article
Affiliations: Department of Mathematics, Statistics, and Computer Science, Saint Olaf College, 1520 Saint Olaf Avenue, Northfield, MN 55057, U.S.A.
Publication date: February 1, 2005