Information geometry on hierarchy of probability distributions. Amari, S. I. IEEE Transactions on Information Theory, 47(5):1701--1711, 2001.
doi  abstract   bibtex   
An exponential family or mixture family of probability$\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\backslash$n“orthogonal” decomposition of such a system based on$\backslash$ninformation geometry. A typical example is the decomposition of$\backslash$nstochastic dependency among a number of random variables. In general,$\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\backslash$neasily represented by correlation, but it is more difficult to measure$\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\backslash$namong these variables. Stochastic dependency is decomposed$\backslash$nquantitatively into an “orthogonal” sum of pairwise,$\backslash$ntriplewise, and further higher order dependencies. This gives a new$\backslash$ninvariant decomposition of joint entropy. This problem is important for$\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\backslash$nneurons and for estimating its functional connections. The orthogonal$\backslash$ndecomposition is given in a wide class of hierarchical structures$\backslash$nincluding both exponential and mixture families. As an example, we$\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\backslash$nthose in various lower order Markov chains
@article{Amari2001,
abstract = {An exponential family or mixture family of probability$\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\backslash$n{\&}ldquo;orthogonal{\&}rdquo; decomposition of such a system based on$\backslash$ninformation geometry. A typical example is the decomposition of$\backslash$nstochastic dependency among a number of random variables. In general,$\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\backslash$neasily represented by correlation, but it is more difficult to measure$\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\backslash$namong these variables. Stochastic dependency is decomposed$\backslash$nquantitatively into an {\&}ldquo;orthogonal{\&}rdquo; sum of pairwise,$\backslash$ntriplewise, and further higher order dependencies. This gives a new$\backslash$ninvariant decomposition of joint entropy. This problem is important for$\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\backslash$nneurons and for estimating its functional connections. The orthogonal$\backslash$ndecomposition is given in a wide class of hierarchical structures$\backslash$nincluding both exponential and mixture families. As an example, we$\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\backslash$nthose in various lower order Markov chains},
author = {Amari, S. I.},
doi = {10.1109/18.930911},
file = {:Users/brekels/Documents/Mendeley Desktop/Information geometry on hierarchy of probability distributions - Amari.pdf:pdf},
issn = {00189448},
journal = {IEEE Transactions on Information Theory},
keywords = {Decomposition of entropy,Extended Pythagoras theorem,Higher order Markov chain,Higher order interactions,Information geometry,Kullback divergence,e- and m-projections},
number = {5},
pages = {1701--1711},
title = {{Information geometry on hierarchy of probability distributions}},
volume = {47},
year = {2001}
}

Downloads: 0