Articles |
Hierarchical testing of variable importance
Department of Statistics, University of Oxford, 1 South Parks Road, Oxford, OX1 3TG, U.K. meinshausen{at}stats.ox.ac.uk
Received for publication 1 August 2006. Revision received 1 October 2007.
A frequently encountered challenge in high-dimensional regression is the detection of relevant variables. Variable selection suffers from instability and the power to detect relevant variables is typically low if predictor variables are highly correlated. When taking the multiplicity of the testing problem into account, the power diminishes even further. To gain power and insight, it can be advantageous to look for influence not at the level of individual variables but rather at the level of clusters of highly correlated variables. We propose a hierarchical approach. Variable importance is first tested at the coarsest level, corresponding to the global null hypothesis. The method then tries to attribute any effect to smaller subclusters or even individual variables. The smallest possible clusters, which still exhibit a significant influence on the response variable, are retained. It is shown that the proposed testing procedure controls the familywise error rate at a prespecified level, simultaneously over all resolution levels. The method has power comparable to the Bonferroni–Holm procedure on the level of individual variables and dramatically larger power for coarser resolution levels. The best resolution level is selected adaptively.
Key Words: Hierarchical clustering • High-dimensional • Higher criticism alternative • Multiple linear regression • Multiple testing
References
-
Ashburner M., Ball C. A., Blake J. A., Botstein D., Butler H., Cherry J. M., Davis A. P., Dolinski K., Dwight S. S., Eppig J. T., Harris M. A., Hill D. P., Issel-Tarver L., Kasarskis A., Lewis S., Matese J. C., Richardson J. E., Ringwald M., Rubin G. M., Sherlock G. Gene ontology: tool for the unification of biology. Nature Genet. (2000) 25:25–9.[CrossRef][Web of Science][Medline]
Benjamini Y., Hochberg Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Statist. Soc. B (1995) 57:289–300.
Benjamini Y., Yekutieli D. The control of the false discovery rate in multiple testing under dependency. Ann. Statist. (2001) 29:1165–88.[CrossRef]
Blanchard G., Geman D. Hierarchical testing designs for pattern recognition. Ann. Statist. (2005) 33:1155–1202.[CrossRef]
Breiman L. Better subset regression using the nonnegative garrote. Technometrics (1995) 37:373–84.[CrossRef][Web of Science]
Breiman L. Statistical modeling: the two cultures. Statist. Sci. (2001) 16:199–215.
Donoho D., Jin J. Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. (2004) 32:962–95.[CrossRef]
Goeman J. J., van de Geer S. A., van Houwelingen H. C. Testing against a high-dimensional alternative. J. R. Statist. Soc. B (2005) 68:477–93.[CrossRef]
Hartigan J. Clustering Algorithms (1975) New York: Wiley.
Hastie T., Tibshirani R., Botstein D., Brown P. Supervised harvesting of expression trees. Genome Biol. (2001) 2:1–12.[Medline]
Holm S. A simple sequentially rejective multiple test procedure. Scand. J. Statist. (1979) 6:65–70.
Meinshausen N. False discovery control for multiple tests of association under general dependence. Scand. J. Statist. (2006) 33:227–37.[CrossRef]
Shaffer J. Modified sequentially rejective multiple test procedures. J. Am. Statist. Assoc. (1986) 81:826–31.[CrossRef][Web of Science]
Shaffer J. Multiple hypothesis testing: a review. Ann. Rev. Psychol. (1995) 46:561–84.[CrossRef][Web of Science]
Ward J. Hierarchical grouping to optimize an objective function. J. Am. Statist. Assoc. (1963) 58:236–44.[CrossRef][Web of Science]
Westfall P., Young S. Resampling-based Multiple Testing: Examples and Methods for p-Value Adjustment (1993) New York: John Wiley & Sons.
Yekutieli D., Reiner-Benaim A., Benjamini Y., Elmer G., Kafkafi N., Letwin N., Lee N. Approaches to multiplicity issues in complex research in microarray analysis. Statist. Neer. (2006) 60:414–37.[CrossRef]
Zou H., Hastie T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. B (2005) 67:301–20.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||