Information geometry on hierarchy of probability distributions

Information geometry on hierarchy of probability distributions. Amari, S. I. IEEE Transactions on Information Theory, 47(5):1701--1711, 2001.
doi abstract bibtex

An exponential family or mixture family of probability$\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\backslash$n“orthogonal” decomposition of such a system based on$\backslash$ninformation geometry. A typical example is the decomposition of$\backslash$nstochastic dependency among a number of random variables. In general,$\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\backslash$neasily represented by correlation, but it is more difficult to measure$\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\backslash$namong these variables. Stochastic dependency is decomposed$\backslash$nquantitatively into an “orthogonal” sum of pairwise,$\backslash$ntriplewise, and further higher order dependencies. This gives a new$\backslash$ninvariant decomposition of joint entropy. This problem is important for$\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\backslash$nneurons and for estimating its functional connections. The orthogonal$\backslash$ndecomposition is given in a wide class of hierarchical structures$\backslash$nincluding both exponential and mixture families. As an example, we$\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\backslash$nthose in various lower order Markov chains

@article{Amari2001,
abstract = {An exponential family or mixture family of probability$\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\backslash$n{\&}ldquo;orthogonal{\&}rdquo; decomposition of such a system based on$\backslash$ninformation geometry. A typical example is the decomposition of$\backslash$nstochastic dependency among a number of random variables. In general,$\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\backslash$neasily represented by correlation, but it is more difficult to measure$\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\backslash$namong these variables. Stochastic dependency is decomposed$\backslash$nquantitatively into an {\&}ldquo;orthogonal{\&}rdquo; sum of pairwise,$\backslash$ntriplewise, and further higher order dependencies. This gives a new$\backslash$ninvariant decomposition of joint entropy. This problem is important for$\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\backslash$nneurons and for estimating its functional connections. The orthogonal$\backslash$ndecomposition is given in a wide class of hierarchical structures$\backslash$nincluding both exponential and mixture families. As an example, we$\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\backslash$nthose in various lower order Markov chains},
author = {Amari, S. I.},
doi = {10.1109/18.930911},
file = {:Users/brekels/Documents/Mendeley Desktop/Information geometry on hierarchy of probability distributions - Amari.pdf:pdf},
issn = {00189448},
journal = {IEEE Transactions on Information Theory},
keywords = {Decomposition of entropy,Extended Pythagoras theorem,Higher order Markov chain,Higher order interactions,Information geometry,Kullback divergence,e- and m-projections},
number = {5},
pages = {1701--1711},
title = {{Information geometry on hierarchy of probability distributions}},
volume = {47},
year = {2001}
}

Downloads: 0

{"_id":"ZCxpK6btzMjDZLHuT","bibbaseid":"amari-informationgeometryonhierarchyofprobabilitydistributions-2001","downloads":0,"creationDate":"2017-03-16T18:57:17.590Z","title":"Information geometry on hierarchy of probability distributions","author_short":["Amari, S. I."],"year":2001,"bibtype":"article","biburl":"https://www.dropbox.com/s/axqr6ulvl7d7meq/library.bib?dl=1","bibdata":{"bibtype":"article","type":"article","abstract":"An exponential family or mixture family of probability$\\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\\backslash$n“orthogonal” decomposition of such a system based on$\\backslash$ninformation geometry. A typical example is the decomposition of$\\backslash$nstochastic dependency among a number of random variables. In general,$\\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\\backslash$neasily represented by correlation, but it is more difficult to measure$\\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\\backslash$namong these variables. Stochastic dependency is decomposed$\\backslash$nquantitatively into an “orthogonal” sum of pairwise,$\\backslash$ntriplewise, and further higher order dependencies. This gives a new$\\backslash$ninvariant decomposition of joint entropy. This problem is important for$\\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\\backslash$nneurons and for estimating its functional connections. The orthogonal$\\backslash$ndecomposition is given in a wide class of hierarchical structures$\\backslash$nincluding both exponential and mixture families. As an example, we$\\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\\backslash$nthose in various lower order Markov chains","author":[{"propositions":[],"lastnames":["Amari"],"firstnames":["S.","I."],"suffixes":[]}],"doi":"10.1109/18.930911","file":":Users/brekels/Documents/Mendeley Desktop/Information geometry on hierarchy of probability distributions - Amari.pdf:pdf","issn":"00189448","journal":"IEEE Transactions on Information Theory","keywords":"Decomposition of entropy,Extended Pythagoras theorem,Higher order Markov chain,Higher order interactions,Information geometry,Kullback divergence,e- and m-projections","number":"5","pages":"1701--1711","title":"Information geometry on hierarchy of probability distributions","volume":"47","year":"2001","bibtex":"@article{Amari2001,\nabstract = {An exponential family or mixture family of probability$\\backslash$ndistributions has a natural hierarchical structure. This paper gives an$\\backslash$n{\\&}ldquo;orthogonal{\\&}rdquo; decomposition of such a system based on$\\backslash$ninformation geometry. A typical example is the decomposition of$\\backslash$nstochastic dependency among a number of random variables. In general,$\\backslash$nthey have a complex structure of dependencies. Pairwise dependency is$\\backslash$neasily represented by correlation, but it is more difficult to measure$\\backslash$neffects of pure triplewise or higher order interactions (dependencies)$\\backslash$namong these variables. Stochastic dependency is decomposed$\\backslash$nquantitatively into an {\\&}ldquo;orthogonal{\\&}rdquo; sum of pairwise,$\\backslash$ntriplewise, and further higher order dependencies. This gives a new$\\backslash$ninvariant decomposition of joint entropy. This problem is important for$\\backslash$nextracting intrinsic interactions in firing patterns of an ensemble of$\\backslash$nneurons and for estimating its functional connections. The orthogonal$\\backslash$ndecomposition is given in a wide class of hierarchical structures$\\backslash$nincluding both exponential and mixture families. As an example, we$\\backslash$ndecompose the dependency in a higher order Markov chain into a sum of$\\backslash$nthose in various lower order Markov chains},\nauthor = {Amari, S. I.},\ndoi = {10.1109/18.930911},\nfile = {:Users/brekels/Documents/Mendeley Desktop/Information geometry on hierarchy of probability distributions - Amari.pdf:pdf},\nissn = {00189448},\njournal = {IEEE Transactions on Information Theory},\nkeywords = {Decomposition of entropy,Extended Pythagoras theorem,Higher order Markov chain,Higher order interactions,Information geometry,Kullback divergence,e- and m-projections},\nnumber = {5},\npages = {1701--1711},\ntitle = {{Information geometry on hierarchy of probability distributions}},\nvolume = {47},\nyear = {2001}\n}\n","author_short":["Amari, S. I."],"key":"Amari2001","id":"Amari2001","bibbaseid":"amari-informationgeometryonhierarchyofprobabilitydistributions-2001","role":"author","urls":{},"keyword":["Decomposition of entropy","Extended Pythagoras theorem","Higher order Markov chain","Higher order interactions","Information geometry","Kullback divergence","e- and m-projections"],"downloads":0,"html":""},"search_terms":["information","geometry","hierarchy","probability","distributions","amari"],"keywords":["decomposition of entropy","extended pythagoras theorem","higher order markov chain","higher order interactions","information geometry","kullback divergence","e- and m-projections"],"authorIDs":[],"dataSources":["5NxYfDAcNJEC35W7g"]}