Article |
A group bridge approach for variable selection
Department of Statistics and Actuarial Science, University of Iowa, 221 Schaeffer Hall, Iowa City, Iowa 52242, U.S.A. jian-huang{at}uiowa.edu
Division of Biostatistics, Department of Epidemiology and Public Health, Yale University, New Haven, Connecticut 06520, U.S.A. shuangge.ma{at}yale.edu
Department of Management Science, University of Miami, Coral Gables, Florida 33124, U.S.A. hxie{at}exchange.sba.miami.edu
Department of Statistics, Rutgers University, Piscataway, New Jersey 08854, U.S.A. cunhui{at}stat.rutgers.edu
Received for publication 1 May 2007. Revision received 1 October 2008.
In multiple regression problems when covariates can be naturally grouped, it is important to carry out feature selection at the group and within-group individual variable levels simultaneously. The existing methods, including the lasso and group lasso, are designed for either variable selection or group selection, but not for both. We propose a group bridge approach that is capable of simultaneous selection at both the group and within-group individual variable levels. The proposed approach is a penalized regularization method that uses a specially designed group bridge penalty. It has the oracle group selection property, in that it can correctly select important groups with probability converging to one. In contrast, the group lasso and group least angle regression methods in general do not possess such an oracle property in group selection. Simulation studies indicate that the group bridge has superior performance in group and individual variable selection relative to several existing methods.
Key Words: Bridge estimator • Iterative lasso • Penalized regression • Two-level selection • Variable-selection consistency
References
-
Akaike H. Information theory and an extension of the maximum likelihood principle. In: Second International Symposium on Information Theory—Petrov B. N., Csaki F., eds. (1973) Budapest: Akademiai Kiado. 267–81.
Efron B., Hastie T., Johnstone I., Tibshirani R. Least angle regression (with Discussion). Ann. Statist. (2004) 32:407–99.[CrossRef]
Fan J., Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc. (2001) 96:1348–60.[CrossRef][Web of Science]
Fan J., Peng H. Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. (2004) 32:928–61.[CrossRef]
Frank I. E., Friedman J. H. A statistical view of some chemometrics regression tools (with Discussion). Technometrics (1993) 35:109–48.[CrossRef][Web of Science]
Fu W. J. Penalized regressions: the bridge versus the Lasso. J. Comp. Graph. Statist. (1998) 7:397–16.[CrossRef]
Huang J., Horowitz J. L., Ma S. G. Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Ann. Statist. (2008) 36:587–13.[CrossRef]
Hunter D. R., Li R. Variable selection using MM algorithms. Ann. Statist. (2005) 33:1617–42.[CrossRef]
Kim Y., Kim J., Kim Y. The blockwise sparse regression. Statist. Sinica (2006) 16:375–90.
Kim J., Pollard D. Cube root asymptotics. Ann. Statist. (1990) 18:191–219.[CrossRef]
Knight K., Fu W. J. Asymptotics for Lasso-type estimators. Ann. Statist. (2000) 28:1356–78.[CrossRef]
Lin Y., Zhang H. Component selection and smoothing in multivariate nonparametric regression. Ann. Statist. (2006) 34:2272–97.[CrossRef]
Ma S., Huang J. Clustering threshold gradient descent regularization: with application to survival analysis using microarray data. Bioinformatics (2007) 23:466–72.
Mallows C. Some comments on Cp. Technometrics (1973) 15:661–75.[CrossRef][Web of Science]
Schwarz G. Estimating the dimension of a model. Ann. Statist. (1978) 6:461–4.[CrossRef]
Storey M. L., Forshee R. A., Weaver A. R., Sansalone W. R. Demographic and lifestyle factors associated with body mass index among children and adolescents. Int. J. Food Sci. Nutr. (2003) 54:491–503.[CrossRef][Web of Science][Medline]
Tibshirani R. Regression shrinkage and selection via the Lasso. J. R. Statist. Soc. (1996) B 58:267–88.
van der Vaart A. W. Asymptotic Statistics (1998) Cambridge: Cambridge University Press.
Wahba G. Spline Models for Observational Data. (1990) CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia, PA: Society for Industrial and Applied Mathematics.
Yang Y. Can the strengths of AICand BICbe shared. Biometrika (2005) 92:937–50.
Yuan M., Lin Y. Model selection and estimation in regression with grouped variables. J. R. Statist. Soc. (2006) B 68:49–67.[CrossRef]
Zhang H. H., Lu W. B. Adaptive Lasso for Cox's proportional hazards model. Biometrika (2007) 94:691–703.
Zou H., Hastie T. Regularization and variable selection via the elastic net. J. R. Statist. Soc. (2005) B 67:301–20.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||