Deep Gaussian Processes. Damianou, A. C. & Lawrence, N. D. Paper abstract bibtex In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.
@article{damianouDeepGaussianProcesses2012,
archivePrefix = {arXiv},
eprinttype = {arxiv},
eprint = {1211.0358},
primaryClass = {cs, math, stat},
title = {Deep {{Gaussian Processes}}},
url = {http://arxiv.org/abs/1211.0358},
abstract = {In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.},
urldate = {2019-04-04},
date = {2012-11-01},
keywords = {Statistics - Machine Learning,Mathematics - Probability,Computer Science - Machine Learning,I.2.6,60G15; 58E30,G.1.2,G.3},
author = {Damianou, Andreas C. and Lawrence, Neil D.},
file = {/home/dimitri/Nextcloud/Zotero/storage/FCZ2J52I/Damianou and Lawrence - 2012 - Deep Gaussian Processes.pdf;/home/dimitri/Nextcloud/Zotero/storage/EPABC27J/1211.html}
}
Downloads: 0
{"_id":"GZ3di7e5wdKj6Rj6L","bibbaseid":"damianou-lawrence-deepgaussianprocesses","authorIDs":[],"author_short":["Damianou, A. C.","Lawrence, N. D."],"bibdata":{"bibtype":"article","type":"article","archiveprefix":"arXiv","eprinttype":"arxiv","eprint":"1211.0358","primaryclass":"cs, math, stat","title":"Deep Gaussian Processes","url":"http://arxiv.org/abs/1211.0358","abstract":"In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.","urldate":"2019-04-04","date":"2012-11-01","keywords":"Statistics - Machine Learning,Mathematics - Probability,Computer Science - Machine Learning,I.2.6,60G15; 58E30,G.1.2,G.3","author":[{"propositions":[],"lastnames":["Damianou"],"firstnames":["Andreas","C."],"suffixes":[]},{"propositions":[],"lastnames":["Lawrence"],"firstnames":["Neil","D."],"suffixes":[]}],"file":"/home/dimitri/Nextcloud/Zotero/storage/FCZ2J52I/Damianou and Lawrence - 2012 - Deep Gaussian Processes.pdf;/home/dimitri/Nextcloud/Zotero/storage/EPABC27J/1211.html","bibtex":"@article{damianouDeepGaussianProcesses2012,\n archivePrefix = {arXiv},\n eprinttype = {arxiv},\n eprint = {1211.0358},\n primaryClass = {cs, math, stat},\n title = {Deep {{Gaussian Processes}}},\n url = {http://arxiv.org/abs/1211.0358},\n abstract = {In this paper we introduce deep Gaussian process (GP) models. Deep GPs are a deep belief network based on Gaussian process mappings. The data is modeled as the output of a multivariate GP. The inputs to that Gaussian process are then governed by another GP. A single layer model is equivalent to a standard GP or the GP latent variable model (GP-LVM). We perform inference in the model by approximate variational marginalization. This results in a strict lower bound on the marginal likelihood of the model which we use for model selection (number of layers and nodes per layer). Deep belief networks are typically applied to relatively large data sets using stochastic gradient descent for optimization. Our fully Bayesian treatment allows for the application of deep models even when data is scarce. Model selection by our variational bound shows that a five layer hierarchy is justified even when modelling a digit data set containing only 150 examples.},\n urldate = {2019-04-04},\n date = {2012-11-01},\n keywords = {Statistics - Machine Learning,Mathematics - Probability,Computer Science - Machine Learning,I.2.6,60G15; 58E30,G.1.2,G.3},\n author = {Damianou, Andreas C. and Lawrence, Neil D.},\n file = {/home/dimitri/Nextcloud/Zotero/storage/FCZ2J52I/Damianou and Lawrence - 2012 - Deep Gaussian Processes.pdf;/home/dimitri/Nextcloud/Zotero/storage/EPABC27J/1211.html}\n}\n\n","author_short":["Damianou, A. C.","Lawrence, N. D."],"key":"damianouDeepGaussianProcesses2012","id":"damianouDeepGaussianProcesses2012","bibbaseid":"damianou-lawrence-deepgaussianprocesses","role":"author","urls":{"Paper":"http://arxiv.org/abs/1211.0358"},"keyword":["Statistics - Machine Learning","Mathematics - Probability","Computer Science - Machine Learning","I.2.6","60G15; 58E30","G.1.2","G.3"],"downloads":0},"bibtype":"article","biburl":"https://raw.githubusercontent.com/dlozeve/newblog/master/bib/all.bib","creationDate":"2020-01-08T20:39:39.316Z","downloads":0,"keywords":["statistics - machine learning","mathematics - probability","computer science - machine learning","i.2.6","60g15; 58e30","g.1.2","g.3"],"search_terms":["deep","gaussian","processes","damianou","lawrence"],"title":"Deep Gaussian Processes","year":null,"dataSources":["3XqdvqRE7zuX4cm8m"]}