Reducing the dimensionality of data with neural networks

Reducing the dimensionality of data with neural networks. Hinton, G. E. & Salakhutdinov, R. R. Science, 313(5786):504-507, July, 2006.

Link

Abstract

Reducing the dimensionality of data with neural networks [pdf]

Paper

Supporting online material

Matlab code doi abstract bibtex

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.

@Article{	  hintonsalakhutdinov2006b,
  abstract	= {High-dimensional data can be converted to low-dimensional
		  codes by training a multilayer neural network with a small
		  central layer to reconstruct high-dimensional input
		  vectors. Gradient descent can be used for fine-tuning the
		  weights in such "autoencoder" networks, but this works well
		  only if the initial weights are close to a good solution.
		  We describe an effective way of initializing the weights
		  that allows deep autoencoder networks to learn
		  low-dimensional codes that work much better than principal
		  components analysis as a tool to reduce the dimensionality
		  of data.},
  added-at	= {2008-07-15T10:05:18.000+0200},
  author	= {Hinton, G. E. and Salakhutdinov, R. R.},
  biburl	= {http://www.bibsonomy.org/bibtex/2135bbce97b449ddf5fca7be88102b53c/tmalsburg}
		  ,
  description	= {Reducing the dimensionality of data with neural
		  ne‥.[Science. 2006] - PubMed Result},
  doi		= {10.1126/science.1127647},
  interhash	= {019918b82518b74f443a22dc58a0117f},
  intrahash	= {135bbce97b449ddf5fca7be88102b53c},
  journal	= {Science},
  keywords	= {dimensionality reduction, neural networks, parameter
		  estimation},
  month		= {July},
  number	= {5786},
  pages		= {504-507},
  pmid		= {16873662},
  title		= {Reducing the dimensionality of data with neural networks},
  url_link	= {http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed&uid=16873662&cmd=showdetailview&indexed=google}
		  ,
  volume	= {313},
  year		= {2006},
  url_abstract	= {http://www.sciencemag.org/cgi/content/abstract/313/5786/504?ijkey=2J.8MGw5yScb2&keytype=ref&siteid=sci}
		  ,
  url_paper	= {http://www.cs.toronto.edu/~hinton/science.pdf},
  url_supporting_online_material={http://www.cs.toronto.edu/~hinton/absps/science_som.pdf}
		  ,
  url_matlab_code={http://www.cs.toronto.edu/~hinton/MatlabForSciencePaper.html}
		  
}

Downloads: 0