Article |
Asymptotic properties of penalized spline estimators
Katholieke Universiteit Leuven, Operations Research & Business Statistics and Leuven Statistics Research Center, Naamsestraat 69, B-3000 Leuven, Belgium gerda.claeskens{at}econ.kuleuven.be
Georg-August-Universität Göttingen, CRC Poverty, Equity and Growth, Platz der Göttinger Sieben 3, D-37073 Göttingen, Germany tatyana.krivobokova{at}wiwi.uni-goettingen.de
Colorado State University Department of Statistics, Fort Collins, Colorado 80523, U.S.A. jopsomer{at}stat.colostate.edu
Received for publication 1 September 2008. Revision received 1 December 2008.
We study the class of penalized spline estimators, which enjoy similarities to both regression splines, without penalty and with fewer knots than data points, and smoothing splines, with knots equal to the data points and a penalty controlling the roughness of the fit. Depending on the number of knots, sample size and penalty, we show that the theoretical properties of penalized regression spline estimators are either similar to those of regression splines or to those of smoothing splines, with a clear breakpoint distinguishing the cases. We prove that using fewer knots results in better asymptotic rates than when using a large number of knots. We obtain expressions for bias and variance and asymptotic rates for the number of knots and penalty parameter.
Key Words: Mean squared error • Nonparametric regression • Penalty • Regression spline • Smoothing spline
References
-
Abramowitz M., Stegun I. Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables (1972) New York: Dover.
Agarwal G., Studden W. Asymptotic integrated mean square error using least squares and bias minimizing splines. Ann. Statist. (1980) 8:1307–25.[CrossRef]
Barrow D. L., Smith P. W. Asymptotic properties of best L2[0, 1] approximation by splines with variable knots. Quart. Appl. Math. (1978) 36:293–304.
Besse P., Cardot H., Ferraty F. Simultaneous nonparametric regression of unbalanced longitudinal data. Comp. Statist. Data Anal. (1997) 24:255–70.[CrossRef]
Bracewell R. The Fourier Transform and Its Applications (1999) New York: McGraw-Hill.
Brumback B. A., Ruppert D., Wand M. P. Comment on Shively, Kohn and Wood. J. Am. Statist. Assoc. (1999) 94:794–97.[CrossRef][Web of Science]
Cardot H. Nonparametric estimation of smoothed principal components analysis of sampled noisy functions. J. Nonparam. Statist. (2000) 12:503–38.[CrossRef]
Carter C. K., Kohn R. Markov chain Monte Carlo in conditionally Gaussian state space models. Biometrika (1996) 83:589–601.
Cox D. D. Asymptotics for M-type smoothing splines. Ann. Statist. (1983) 11:530–51.[CrossRef]
Craven P., Wahba G. Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the method of generalized cross-validation. Numer. Math. (1978) 31:377–403.[CrossRef]
de Boor C. A Practical Guide to Splines (2001) revised ed. New York: Springer.
Demko S. Inverses of band matrices and local convergence of spline projections. SIAM. J. Numer. Anal. (1977) 14:616–9.[CrossRef]
Demmler A., Reinsch C. Oscillation matrices with spline smoothing. Numer. Math. (1975) 24:375–82.[CrossRef]
Eilers P. H. C., Marx B. D. Flexible smoothing with B-splines and penalties. Statist. Sci. (1996) 11:89–121. (with comments and a rejoinder by the authors).[CrossRef]
Eubank R. L. Nonparametric Regression and Spline Smoothing (1999) 2nd ed. New York: Marcel Dekker. Statistics: Textbooks and Monographs 157.
Green P. J., Silverman B. W. Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. Monographs on Statistics and Applied Probability 58 (1994) London: Chapman & Hall.
Hall P., Opsomer J. Theory for penalized spline regression. Biometrika (2005) 92:105–18.
Huang J. Z. Asymptotics for polynomial spline regression under weak conditions. Statist. Prob. Lett. (2003a) 65:207–16.[CrossRef]
Huang J. Z. Local asymptotics for polynomial spline regression. Ann. Statist. (2003b) 31:1600–35.[CrossRef]
Kauermann G., Krivobokova T., Fahrmeir L. Some asymptotic results on generalized penalized spline smoothing. J. R. Statist. Soc. B 71:487–503.
Kelly C., Rice J. Monotone smoothing with application to dose-response curves and the assessment of synergism. Biometrics (1990) 46:1071–85.[CrossRef][Web of Science][Medline]
Li Y., Ruppert D. On the asymptotics of penalized splines. Biometrika (2008) 95:415–36.
Lu L.-Z., Pearce C. Some new bounds for singular values and eigenvalues of matrix products. Ann. Oper. Res. (2000) 98:141–48.[CrossRef]
Nychka D. Splines as local smoothers. Ann. Statist. (1995) 23:1175–97.[CrossRef]
Oehlert G. W. Relaxed boundary smoothing splines. Ann. Statist. (1992) 20:146–60.[CrossRef]
O'Sullivan F. A statistical perspective on ill-posed inverse problems. Statist. Sci. (1986) 1:505–27. With discussion.
Rice J., Rosenblatt M. Integrated mean squared error of a smoothing spline. J. Approx. Theory (1981) 33:353–69.[CrossRef]
Rice J., Rosenblatt M. Smoothing splines: regression, derivatives and deconvolution. Ann. Statist. (1983) 11:141–56.[CrossRef]
Ruppert D., Carroll R. Spatially-adaptive penalties for spline fitting. Aust. New Zeal. J. Statist. (2000) 42:205–24.[CrossRef]
Ruppert D., Wand M., Carroll R. Semiparametric Regression (2003) Cambridge, UK: Cambridge University Press.
Schumaker L. L. Spline Functions: Basic Theory (1981) New York: Wiley.
Schwetlick H., Kunert V. Spline smoothing under constraints on derivatives. BIT (1993) 33:512–28.[CrossRef][Web of Science]
Speckman P. Spline smoothing and optimal rates of convergence in nonparametric regression models. Ann. Statist. (1985) 13:970–83.[CrossRef]
Speckman P. L., Sun D. Fully Bayesian spline smoothing and intrinsic autoregressive priors. Biometrika (2003) 90:289–302.
Speed T. Comment on "that BLUP is a good thing: The estimation of random effects," by G. K. Robinson. Statist. Sci. (1991) 6:42–44.[CrossRef]
Stone C. J. Optimal rate of convergence for nonparametric regression. Ann. Statist. (1982) 10:1040–53.[CrossRef]
Utreras F. Sur le choix du paramètre d'ajustement dans le lissage par fonctions spline. Numer. Math. (1980) 34:15–28.[CrossRef]
Utreras F. Optimal smoothing of noisy data using spline functions. SIAM J. Sci. Statist. Comp. (1981) 2:349–62.[CrossRef]
Utreras F. Natural spline functions, their associated eigenvalue problem. Numer. Math. (1983) 42:107–17.[CrossRef]
Utreras F. Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates. Numer. Math. (1985) 47:611–25.[CrossRef]
Utreras F. Boundary effects on convergence rates for Tikhonov regularization. J. Approx. Theory (1988) 54:235–49.[CrossRef]
Wahba G. Smoothing noisy data with spline functions. Numer. Math. (1975) 24:383–93.[CrossRef]
Wahba G. Spline Models for Observational Data (1990) Philadelphia, PA: SIAM. CBMS-NSF Regional Conference Series in Applied Mathematics 59.
Wand M., Ormerod J. On semiparametric regression with O'Sullivan penalised splines. Aust. New Zeal. J. Statist. (2008) 50:179–98.[CrossRef]
Zhou S., Shen X., Wolfe D. A. Local asymptotics for regression splines and confidence regions. Ann. Statist. (1998) 26:1760–82.[CrossRef]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||