© 2001 by Biometrika Trust
Bayesian inference in neural networks
1 Department of Mathematics and Statistics, Texas Technological University, Lubbock, Texas 79409, U.S.Arpaige{at}math.ttu.edu 2 Department of Statistics, Colorado State University, Fort Collins, Colorado 80523, U.S.A.walrus{at}stat.colostate.edu
Approximate marginal Bayesian computation and inference are developed for neural network models.The marginal considerations include determination of approximate Bayes factors for model choice about the number of nonlinear sigmoid terms, approximate predictive density computation for a future observable and determination of approximate Bayes estimates for the nonlinear regression function. Standard conjugate analysis applied to the linear parameters leads to an explicit posterior on the nonlinear parameters. Further marginalisation is performed using Laplace approximations. The choice of prior and the use of an alternative sigmoid lead to posterior invariance in the nonlinear parameter which is discussed in connection with the lack of sigmoid identifiability. A principal finding is that parsimonious model choice is best determined from the list of modal estimates used in the Laplace approximation of the Bayes factors for various numbers of sigmoids. By comparison, the values of the various Bayes factors are of only secondary importance. The proposed methods are illustrated in the context of two nonlinear datasets that involve respectively univariate and multivariate nonlinear regression models.
Key Words: Bayesian computation; Laplace approximation; Model choice; Neural network; Prediction
Received April 1999. Revised February 2001