Stochastic Backpropagation and Approximate Inference in Deep Generative Models

Stochastic Backpropagation and Approximate Inference in Deep Generative Models. Rezende, D. J., Mohamed, S., & Wierstra, D.

Paper abstract bibtex

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation – rules for back-propagation through stochastic variables – and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.

@article{rezendeStochasticBackpropagationApproximate2014,
  archivePrefix = {arXiv},
  eprinttype = {arxiv},
  eprint = {1401.4082},
  primaryClass = {cs, stat},
  title = {Stochastic {{Backpropagation}} and {{Approximate Inference}} in {{Deep Generative Models}}},
  url = {http://arxiv.org/abs/1401.4082},
  abstract = {We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.},
  urldate = {2019-01-25},
  date = {2014-01-16},
  keywords = {Statistics - Machine Learning,Computer Science - Artificial Intelligence,Statistics - Computation,Statistics - Methodology,Computer Science - Machine Learning},
  author = {Rezende, Danilo Jimenez and Mohamed, Shakir and Wierstra, Daan},
  file = {/home/dimitri/Nextcloud/Zotero/storage/KQQEGKHT/Rezende et al. - 2014 - Stochastic Backpropagation and Approximate Inferen.pdf;/home/dimitri/Nextcloud/Zotero/storage/DHSBCQPC/1401.html}
}

Downloads: 0

{"_id":"2vTbLj3uP6spZGTuF","bibbaseid":"rezende-mohamed-wierstra-stochasticbackpropagationandapproximateinferenceindeepgenerativemodels","authorIDs":[],"author_short":["Rezende, D. J.","Mohamed, S.","Wierstra, D."],"bibdata":{"bibtype":"article","type":"article","archiveprefix":"arXiv","eprinttype":"arxiv","eprint":"1401.4082","primaryclass":"cs, stat","title":"Stochastic Backpropagation and Approximate Inference in Deep Generative Models","url":"http://arxiv.org/abs/1401.4082","abstract":"We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation – rules for back-propagation through stochastic variables – and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.","urldate":"2019-01-25","date":"2014-01-16","keywords":"Statistics - Machine Learning,Computer Science - Artificial Intelligence,Statistics - Computation,Statistics - Methodology,Computer Science - Machine Learning","author":[{"propositions":[],"lastnames":["Rezende"],"firstnames":["Danilo","Jimenez"],"suffixes":[]},{"propositions":[],"lastnames":["Mohamed"],"firstnames":["Shakir"],"suffixes":[]},{"propositions":[],"lastnames":["Wierstra"],"firstnames":["Daan"],"suffixes":[]}],"file":"/home/dimitri/Nextcloud/Zotero/storage/KQQEGKHT/Rezende et al. - 2014 - Stochastic Backpropagation and Approximate Inferen.pdf;/home/dimitri/Nextcloud/Zotero/storage/DHSBCQPC/1401.html","bibtex":"@article{rezendeStochasticBackpropagationApproximate2014,\n archivePrefix = {arXiv},\n eprinttype = {arxiv},\n eprint = {1401.4082},\n primaryClass = {cs, stat},\n title = {Stochastic {{Backpropagation}} and {{Approximate Inference}} in {{Deep Generative Models}}},\n url = {http://arxiv.org/abs/1401.4082},\n abstract = {We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.},\n urldate = {2019-01-25},\n date = {2014-01-16},\n keywords = {Statistics - Machine Learning,Computer Science - Artificial Intelligence,Statistics - Computation,Statistics - Methodology,Computer Science - Machine Learning},\n author = {Rezende, Danilo Jimenez and Mohamed, Shakir and Wierstra, Daan},\n file = {/home/dimitri/Nextcloud/Zotero/storage/KQQEGKHT/Rezende et al. - 2014 - Stochastic Backpropagation and Approximate Inferen.pdf;/home/dimitri/Nextcloud/Zotero/storage/DHSBCQPC/1401.html}\n}\n\n","author_short":["Rezende, D. J.","Mohamed, S.","Wierstra, D."],"key":"rezendeStochasticBackpropagationApproximate2014","id":"rezendeStochasticBackpropagationApproximate2014","bibbaseid":"rezende-mohamed-wierstra-stochasticbackpropagationandapproximateinferenceindeepgenerativemodels","role":"author","urls":{"Paper":"http://arxiv.org/abs/1401.4082"},"keyword":["Statistics - Machine Learning","Computer Science - Artificial Intelligence","Statistics - Computation","Statistics - Methodology","Computer Science - Machine Learning"],"downloads":0},"bibtype":"article","biburl":"https://raw.githubusercontent.com/dlozeve/newblog/master/bib/all.bib","creationDate":"2020-01-08T20:39:39.236Z","downloads":0,"keywords":["statistics - machine learning","computer science - artificial intelligence","statistics - computation","statistics - methodology","computer science - machine learning"],"search_terms":["stochastic","backpropagation","approximate","inference","deep","generative","models","rezende","mohamed","wierstra"],"title":"Stochastic Backpropagation and Approximate Inference in Deep Generative Models","year":null,"dataSources":["3XqdvqRE7zuX4cm8m"]}