© 2004 by Biometrika Trust
Bayesian criterion based model assessment for categorical data
1 Department of Statistics, University of Connecticut, 215 Glenbrook Road, U-4120, Storrs, Connecticut 06269, U.S.Amhchen{at}stat.uconn.edu dey{at}stat.uconn.edu 2 Department of Biostatistics, University of North Carolina, McGavran Greenberg Hall, CB#7420, Chapel Hill, North Carolina 27599, U.S.A.ibrahim{at}bios.unc.edu
We propose a general Bayesian criterion for model assessment for categorical data called the weighted L measure, which is constructed from the posterior predictive distribution of the data.The measure is based on weighting the observations according to the sampling variance of their future response vector. The weight component in the weighted L measure plays the role of a penalty term in the criterion, in which a greater weight assigned to covariate values implies a greater penalty term on the dimension of the model. A detailed justification is provided for such a weighting procedure and several theoretical properties of the weighted L measure are presented for a wide variety of discrete data models. For these models, we examine properties of the weighted L measure, and show that it can perform better than the unweighted L measure in a variety of settings. In addition, we show that the weighted quadratic loss L measure is more attractive than the unweighted L measure and the deviance loss L measure for categorical data. Moreover, a calibration for the weighted L measure is motivated and proposed, which allows us to compare formally the L measure values of competing models. A detailed simulation study is presented to examine the performance of the weighted L measure, and it is compared to other established model-selection methods. Finally, the method is applied to a real dataset using a bivariate ordinal response model.
Key Words: Binary data; L measure; Loss function; Model selection; Multivariate categorical response; Ordinal regression; Weighted L measure
Received November 2001. Revised May 2003