Biometrika Advance Access originally published online on August 5, 2007
Biometrika 2007 94(3):553-568; doi:10.1093/biomet/asm053
Copyright © 2007 Biometrika Trust
Articles |
Tuning parameter selectors for the smoothly clipped absolute deviation method
Guanghua School of Management, Peking University, Beijing, China, 100871
Department of Statistics and The Methodology Center, The Pennsylvania State University, University Park, Pennsylvania 16802-2111, U.S.A.
Graduate School of Management, University of California, Davis, California 95616-8609, U.S.A.
hansheng{at}gsm.pku.edu.cn
rli{at}stat.psu.edu
cltsai{at}ucdavis.edu
Received for publication 1 November 2005. Revision received 1 November 2006.
The penalized least squares approach with smoothly clipped absolute deviation penalty has been consistently demonstrated to be an attractive regression shrinkage and selection method. It not only automatically and consistently selects the important variables, but also produces estimators which are as efficient as the oracle estimator. However, these attractive features depend on appropriate choice of the tuning parameter. We show that the commonly used generalized crossvalidation cannot select the tuning parameter satisfactorily, with a nonignorable overfitting effect in the resulting model. In addition, we propose a BIC tuning parameter selector, which is shown to be able to identify the true model consistently. Simulation studies are presented to support theoretical findings, and an empirical example is given to illustrate its use in the Female Labor Supply data.
Key Words: AIC • BIC • Generalized crossvalidation • Least absolute shrinkage and selection operator • Smoothly clipped absolute deviation
References
-
Akaike H. Information theory and an extension of the maximum likelihood principle. Petrov B. N., Csaki F., eds. (1973) Proceedings of the 2nd International Symposium Information Theory. Budapest: Akademia Kiado. 267–81.
Breiman L. Heuristics of instability and stabilization in model selection. Ann. Statist. (1996) 24:2350–3.[CrossRef]
Bunea F. Consistent covariate selection and post model selection inference in semiparametric regression. Ann. Statist. (2004) 32:898–927.[CrossRef]
Bunea F., Wegkamp M. Two-stage model selection procedures in partially linear regression. Can. J. Statist. (2004) 32:105–118.
Craven P., Wahba G. Smoothing noisy data with spline function: Estimating the correct degree of smoothing by the method of generalized cross validation. Numer. Math. (1979) 31:337–403.
Engle R. F., Granger C. W. J., Rice J., Weiss A. Semiparametric estimates of the relation between weather and electricity sales. J. Am. Statist. Assoc. (1986) 81:310–20.[CrossRef][ISI]
Fan J., Gijbels I. Local Polynomial Modelling and Its Applications (1996) New York: Chapman and Hall.
Fan J., Härdle W., Mammen E. Direct estimation of low-dimensional components in additive models. Ann. Statist. (1998) 26:943–71.[CrossRef]
Fan J., Huang T. Profile likelihood inference on semiparametric varying-coefficient partially linear models. Bernoulli (2005) 11:1031–57.[ISI]
Fan J., Li R. Variable selection via nonconcave penalised likelihood and its oracle properties. J. Am. Statist. Assoc. (2001) 96:1348–60.[CrossRef][ISI]
Fan J., Li R. New estimation and model selection procedures for semiparametric modeling in longitudinal data analysis. J. Am. Statist. Assoc. (2004) 99:710–23.[CrossRef][ISI]
Härdle W., Liang H., Gao J. Partially Linear Models (2000) Heidelberg: Springer Physica-Verlag.
Heckman N. E. Spline smoothing in partly linear models. J. R. Statist. Soc. (1986) B 48:244–8.
Huang J., Yang L. Identification of non-linear additive autoregressive models. J. R. Statist. Soc. (2004) B 66:463–77.[CrossRef]
Mack Y. P., Silverman B. W. Weak and strong uniform consistency of kernel regression estimates. Z. Wahr. Verw. Geb. (1982) 61:405–15.[CrossRef]
McQuarrie D. R., Tsai C. L. Regression and Time Series Model Selection (1998) Singapore: World Scientific.
Robinson P. M. Root-n-consistent semiparametric regression. Econometrica (1988) 56:931–54.[CrossRef][ISI]
Ruppert D., Sheather S. J., Wand M. P. An effective bandwidth selector for local least squares regression. J. Am. Statist. Assoc. (1995) 90:1257–70.[CrossRef][ISI]
Schwarz G. Estimating the dimension of a model. Ann. Statist. (1978) 6:461–4.
Shao J. An asymptotic theory for linear model selection. Statist. Sinica (1997) 7:221–64.
Shi P., Tsai C. L. Regression model selection–a residual likelihood approach. J. R. Statist. Soc. (2002) B 64:237–52.[CrossRef]
Shi P., Tsai C. L. A joint regression variable and autoregressive order selection criterion. J. Time Ser. Anal. (2004) 25:923–41.[CrossRef]
Speckman P. Kernel smoothing in partially linear models. J. R. Statist. Soc. (1988) B 50:413–36.
Tibshirani R. J. Regression shrinkage and selection via the lasso. J. R. Statist. Soc. (1996) B 58:267–88.
Yang Y. Can the strengths of AIC and BIC be shared? A conflict between model identification and regression estimation. Biometrika (2005) 92:973–50.
Yatchew A. An elementary estimator for the partially linear model. Economet. Lett. (1997) 57:135–43.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||