Plots of P-values to evaluate many tests simultaneously

doi:10.1093/biomet/69.3.493

Biometrika 1982 69(3):493-502; doi:10.1093/biomet/69.3.493
© 1982 by Biometrika Trust

This Article

	Full Text (PDF)
	Alert me when this article is cited
	Alert me if a correction is posted

Services

	Email this article to a friend
	Similar articles in this journal
	Alert me to new issues of the journal
	Add to My Personal Archive
	Download to citation manager
	Request Permissions

Google Scholar

	Articles by SCHWEDER, T.
	Articles by SPJØTVOLL, E.
	Search for Related Content

Social Bookmarking

	What's this?

Plots of P-values to evaluate many tests simultaneously

T. SCHWEDER and E. SPJØTVOLL

Institute of Mathematical and Physical Sciences, University of Tromse Norway
Department of Mathematics and Statistics, Agricultural University of Norway Aas, Norway

When a large number of tests are made, possibly on the samedata, it is proposed to base a simultaneous evaluation of allthe tests on a plot of cumulative P-values using the observedsignificance probabilities. The points corresponding to truenull hypotheses should form a straight line, while those forfalse null hypotheses should deviate from this line. The linemay be used to estimate the number of true hypotheses. The propertiesof the method are studied in some detail for the problems ofcomparing all pairs of means in a one-way layout, testing allcorrelation coefficients in a large correlation matrix, andthe evaluation of all 2×2 subtables in a contingency table.The plot is illustrated on real data.

Key Words: Simultaneous tests • Multiple comparison • P-value plot • One-way layout • Correlation matrix • Contingency table

Add to CiteULike CiteULike Add to Connotea Connotea Add to Del.icio.us Del.icio.us What's this?

This article has been cited by other articles:

W. Guo
A note on adaptive Bonferroni and Holm procedures under dependence
Biometrika, December 1, 2009; 96(4): 1012 - 1018.
[Abstract] [PDF]

T. Oura, S. Matsui, and K. Kawakami
Sample size calculations for controlling the distribution of false discovery proportion in microarray experiments
Biostat., October 1, 2009; 10(4): 694 - 705.
[Abstract] [Full Text] [PDF]

S. S. Young, H. Bang, and K. Oktay
Cereal-induced gender selection? Most likely a multiple testing false positive
Proc R Soc B, April 7, 2009; 276(1660): 1211 - 1212.
[Full Text] [PDF]

A. Farcomeni
A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion
Statistical Methods in Medical Research, August 1, 2008; 17(4): 347 - 388.
[Abstract] [PDF]

K. Strimmer
fdrtool: a versatile R package for estimating local and tail area-based false discovery rates
Bioinformatics, June 15, 2008; 24(12): 1461 - 1462.
[Abstract] [Full Text] [PDF]

T. S. Mehta, S. O. Zakharkin, G. L. Gadbury, and D. B. Allison
Epistemological issues in omics and high-dimensional biology: give the people what they want
Physiol Genomics, December 13, 2006; 28(1): 24 - 32.
[Abstract] [Full Text] [PDF]

K. F. Manly, D. Nettleton, and J.T. G. Hwang
Genomics, Prior Probability, and Statistical Tests of Multiple Hypotheses
Genome Res., June 1, 2004; 14(6): 997 - 1001.
[Full Text] [PDF]

R. L. Fernando, D. Nettleton, B. R. Southey, J. C. M. Dekkers, M. F. Rothschild, and M. Soller
Controlling the Proportion of False Positives in Multiple Dependent Tests
Genetics, January 1, 2004; 166(1): 611 - 619.
[Abstract] [Full Text] [PDF]

N. J. Schork, J. P. Gardner, L. Zhang, D. Fallin, B. Thiel, H. Jakubowski, and A. Aviv
Genomic Association/Linkage of Sodium Lithium Countertransport in CEPH Pedigrees
Hypertension, November 1, 2002; 40(5): 619 - 628.
[Abstract] [Full Text] [PDF]

C. C. Sheridan, D. K. Steinberg, and G. W. Kling
The microbial and metazoan community associated with colonies of Trichodesmium spp.: a quantitative survey
J. Plankton Res., September 1, 2002; 24(9): 913 - 922.
[Abstract] [Full Text] [PDF]

D. C. Thomas and J. S. Witte
Point: Population Stratification: A Problem for Case-Control Studies of Candidate-Gene Associations?
Cancer Epidemiol. Biomarkers Prev., June 1, 2002; 11(6): 505 - 512.
[Full Text] [PDF]

X. Gao, G. W. Nelson, P. Karacki, M. P. Martin, J. Phair, R. Kaslow, J. J. Goedert, S. Buchbinder, K. Hoots, D. Vlahov, et al.
Effect of a Single Amino Acid Change in MHC Class I Molecules on the Rate of Progression to AIDS
N. Engl. J. Med., May 31, 2001; 344(22): 1668 - 1675.
[Abstract] [Full Text] [PDF]

H. D. Shin, C. Winkler, J. C. Stephens, J. Bream, H. Young, J. J. Goedert, T. R. O'Brien, D. Vlahov, S. Buchbinder, J. Giorgi, et al.
Genetic restriction of HIV-1 pathogenesis to AIDS by promoter alleles of IL10
PNAS, December 19, 2000; 97(26): 14467 - 14472.
[Abstract] [Full Text] [PDF]

Y. Benjamini and Y. Hochberg
On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics
Journal of Educational and Behavioral Statistics, January 1, 2000; 25(1): 60 - 83.
[Abstract] [PDF]

D. J. Schaid, K. Buetow, D. E. Weeks, E. Wijsman, S.-W. Guo, J. Ott, and C. Dahl
Discovery of Cancer Susceptibility Genes: Study Designs, Analytic Approaches, and Trends in Technology
J Natl Cancer Inst Monographs, December 1, 1999; 1999(26): 1 - 16.
[Abstract] [Full Text]

M. Carrington, G. W. Nelson, M. P. Martin, T. Kissner, D. Vlahov, J. J. Goedert, R. Kaslow, S. Buchbinder, K. Hoots, and S. J. O'Brien
HLA and HIV-1: Heterozygote Advantage and B*35-Cw*04 Disadvantage
Science, March 12, 1999; 283(5408): 1748 - 1752.
[Abstract] [Full Text]

E. I. Ginns, P. St. Jean, R. A. Philibert, M. Galdzicka, P. Damschroder-Williams, B. Thiel, R. T. Long, L. J. Ingraham, H. Dalwaldi, M. A. Murray, et al.
A genome-wide search for chromosomal loci linked to mental health wellness in relatives at high risk for bipolar affective disorder among the Old Order Amish
PNAS, December 22, 1998; 95(26): 15531 - 15536.
[Abstract] [Full Text] [PDF]

M. P. Martin, M. Dean, M. W. Smith, C. Winkler, B. Gerrard, N. L. Michael, B. Lee, R. W. Doms, J. Margolick, S. Buchbinder, et al.
Genetic Acceleration of AIDS Progression by a Promoter Variant of CCR5
Science, December 4, 1998; 282(5395): 1907 - 1911.
[Abstract] [Full Text]

M. E. Pichichero, M. A. Deloria, M. B. Rennels, E. L. Anderson, K. M. Edwards, M. D. Decker, J. A. Englund, M. C. Steinhoff, A. Deforest, and B. D. Meade
A Safety and Immunogenicity Comparison of 12 Acellular Pertussis Vaccines and One Whole-Cell Pertussis Vaccine Given as a Fourth Dose in 15- to 20-Month-Old Children
Pediatrics, November 1, 1997; 100(5): 772 - 788.
[Abstract] [Full Text] [PDF]

I. GRANT, J. H. ATKINSON, J. R. HESSELINK, C. J. KENNEDY, D. D. RICHMAN, S. A. SPECTOR, and J. A. McCUTCHAN
Evidence for Early Central Nervous System Involvement in the Acquired Immunodeficiency Syndrome (AIDS) and Other Human Immunodeficiency Virus (HIV) Infections: Studies with Neuropsychologic Testing and Magnetic Resonance Imaging
Ann Intern Med, December 1, 1987; 107(6): 828 - 836.
[Abstract] [PDF]

				T. Oura, S. Matsui, and K. Kawakami Sample size calculations for controlling the distribution of false discovery proportion in microarray experiments Biostat., October 1, 2009; 10(4): 694 - 705. [Abstract] [Full Text] [PDF]

				S. S. Young, H. Bang, and K. Oktay Cereal-induced gender selection? Most likely a multiple testing false positive Proc R Soc B, April 7, 2009; 276(1660): 1211 - 1212. [Full Text] [PDF]

				A. Farcomeni A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion Statistical Methods in Medical Research, August 1, 2008; 17(4): 347 - 388. [Abstract] [PDF]

				K. Strimmer fdrtool: a versatile R package for estimating local and tail area-based false discovery rates Bioinformatics, June 15, 2008; 24(12): 1461 - 1462. [Abstract] [Full Text] [PDF]

				T. S. Mehta, S. O. Zakharkin, G. L. Gadbury, and D. B. Allison Epistemological issues in omics and high-dimensional biology: give the people what they want Physiol Genomics, December 13, 2006; 28(1): 24 - 32. [Abstract] [Full Text] [PDF]

				K. F. Manly, D. Nettleton, and J.T. G. Hwang Genomics, Prior Probability, and Statistical Tests of Multiple Hypotheses Genome Res., June 1, 2004; 14(6): 997 - 1001. [Full Text] [PDF]

				R. L. Fernando, D. Nettleton, B. R. Southey, J. C. M. Dekkers, M. F. Rothschild, and M. Soller Controlling the Proportion of False Positives in Multiple Dependent Tests Genetics, January 1, 2004; 166(1): 611 - 619. [Abstract] [Full Text] [PDF]

				N. J. Schork, J. P. Gardner, L. Zhang, D. Fallin, B. Thiel, H. Jakubowski, and A. Aviv Genomic Association/Linkage of Sodium Lithium Countertransport in CEPH Pedigrees Hypertension, November 1, 2002; 40(5): 619 - 628. [Abstract] [Full Text] [PDF]

				C. C. Sheridan, D. K. Steinberg, and G. W. Kling The microbial and metazoan community associated with colonies of Trichodesmium spp.: a quantitative survey J. Plankton Res., September 1, 2002; 24(9): 913 - 922. [Abstract] [Full Text] [PDF]

				D. C. Thomas and J. S. Witte Point: Population Stratification: A Problem for Case-Control Studies of Candidate-Gene Associations? Cancer Epidemiol. Biomarkers Prev., June 1, 2002; 11(6): 505 - 512. [Full Text] [PDF]

				X. Gao, G. W. Nelson, P. Karacki, M. P. Martin, J. Phair, R. Kaslow, J. J. Goedert, S. Buchbinder, K. Hoots, D. Vlahov, et al. Effect of a Single Amino Acid Change in MHC Class I Molecules on the Rate of Progression to AIDS N. Engl. J. Med., May 31, 2001; 344(22): 1668 - 1675. [Abstract] [Full Text] [PDF]

				H. D. Shin, C. Winkler, J. C. Stephens, J. Bream, H. Young, J. J. Goedert, T. R. O'Brien, D. Vlahov, S. Buchbinder, J. Giorgi, et al. Genetic restriction of HIV-1 pathogenesis to AIDS by promoter alleles of IL10 PNAS, December 19, 2000; 97(26): 14467 - 14472. [Abstract] [Full Text] [PDF]

				Y. Benjamini and Y. Hochberg On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics Journal of Educational and Behavioral Statistics, January 1, 2000; 25(1): 60 - 83. [Abstract] [PDF]

				D. J. Schaid, K. Buetow, D. E. Weeks, E. Wijsman, S.-W. Guo, J. Ott, and C. Dahl Discovery of Cancer Susceptibility Genes: Study Designs, Analytic Approaches, and Trends in Technology J Natl Cancer Inst Monographs, December 1, 1999; 1999(26): 1 - 16. [Abstract] [Full Text]

				M. Carrington, G. W. Nelson, M. P. Martin, T. Kissner, D. Vlahov, J. J. Goedert, R. Kaslow, S. Buchbinder, K. Hoots, and S. J. O'Brien *HLA and HIV-1: Heterozygote Advantage and B35-Cw04 Disadvantage* Science, March 12, 1999; 283(5408): 1748 - 1752. [Abstract] [Full Text]

				E. I. Ginns, P. St. Jean, R. A. Philibert, M. Galdzicka, P. Damschroder-Williams, B. Thiel, R. T. Long, L. J. Ingraham, H. Dalwaldi, M. A. Murray, et al. A genome-wide search for chromosomal loci linked to mental health wellness in relatives at high risk for bipolar affective disorder among the Old Order Amish PNAS, December 22, 1998; 95(26): 15531 - 15536. [Abstract] [Full Text] [PDF]

				M. P. Martin, M. Dean, M. W. Smith, C. Winkler, B. Gerrard, N. L. Michael, B. Lee, R. W. Doms, J. Margolick, S. Buchbinder, et al. Genetic Acceleration of AIDS Progression by a Promoter Variant of CCR5 Science, December 4, 1998; 282(5395): 1907 - 1911. [Abstract] [Full Text]

				M. E. Pichichero, M. A. Deloria, M. B. Rennels, E. L. Anderson, K. M. Edwards, M. D. Decker, J. A. Englund, M. C. Steinhoff, A. Deforest, and B. D. Meade A Safety and Immunogenicity Comparison of 12 Acellular Pertussis Vaccines and One Whole-Cell Pertussis Vaccine Given as a Fourth Dose in 15- to 20-Month-Old Children Pediatrics, November 1, 1997; 100(5): 772 - 788. [Abstract] [Full Text] [PDF]

				I. GRANT, J. H. ATKINSON, J. R. HESSELINK, C. J. KENNEDY, D. D. RICHMAN, S. A. SPECTOR, and J. A. McCUTCHAN Evidence for Early Central Nervous System Involvement in the Acquired Immunodeficiency Syndrome (AIDS) and Other Human Immunodeficiency Virus (HIV) Infections: Studies with Neuropsychologic Testing and Magnetic Resonance Imaging Ann Intern Med, December 1, 1987; 107(6): 828 - 836. [Abstract] [PDF]