© 1994 by Biometrika Trust
MISCELLANEA |
Local divergence and association
Department of Mathematics, Imperial College of Science, Technology of Medicine 180 Queens Gate, London SW7 2BZ, U. K.
Received for publication 1 July 1993.
Revision received 1 March 1994.
 |
Abstract |
|---|
In applications of differential geometry to problems of parametric statistical inference, the notion of divergence is often used to measure the separation between two parametric densities. In this paper we move away from the parametric framework of much of statistical differential geometry and instead employ divergence in a covariate setting. Many data-analyses involve investigation of how the conditional distribution f(y|X = x) of a random variable Y changes with covariate values 
. We propose using local divergence between conditional distributions as a measure of this change and thus as a general measure of association between Y and the covariates X. Under certain regularity conditions we define a class of divergences which are locally the Rao distance (I, R). The limiting LR divergence is bounded below by the signal-to-noise ratio, with equality holding if and only if the conditional density comes from the natural exponential family. The correlation curve and in particular the correlation coefficient are simply transformations of the signal-to-noise ratio and thus are transformations of local divergence. We therefore obtain a differential geometric interpretation of standard ideas of association. The class of LR divergences is broad and includes the Kullback-Leibler divergence and Renyi 
-information measures. We can therefore interpret local association as local utility or information gain of order . We obtain comparable results without regularity conditions.
Key Words: Alpha-divergence • Chi-squared divergence • Correlation curve • Exponential family • Fisher information • Local association • Local divergence • Rao distance • Signal-to-noise ratio • Utility