Learning multiple layers of representation

Learning multiple layers of representation. Hinton, G. E. Trends in Cognitive Sciences, 2007.

To achieve its' impressive performance at tasks such as speech or object recognition, the brain extracts multiple levels of representation from the sensory input. Backpropagation was the ﬁrst computation- ally eﬃcient model of how neural networks could learn multiple layers of representation, but it required labeled training data and it did not work well in deep networks. The limitations of backpropagation learn- ing can now be overcome by using multi-layer neural networks that contain top-down connections and training them to generate sensory data rather than to classify it. Learning multilayer generative models appears to be diﬃcult, but a recent discovery makes it easy to learn non-linear, distributed representations one layer at a time. The mul- tiple layers of representation learned in this way can subsequently be ﬁne-tuned to produce generative or discriminative models that work much better than previous approaches.

@Article{	  hinton2007:learninglayers,
  abstract	= {To achieve its' impressive performance at tasks such as
		  speech or object recognition, the brain extracts multiple
		  levels of representation from the sensory input.
		  Backpropagation was the ﬁrst computation- ally eﬃcient
		  model of how neural networks could learn multiple layers of
		  representation, but it required labeled training data and
		  it did not work well in deep networks. The limitations of
		  backpropagation learn- ing can now be overcome by using
		  multi-layer neural networks that contain top-down
		  connections and training them to generate sensory data
		  rather than to classify it. Learning multilayer generative
		  models appears to be diﬃcult, but a recent discovery
		  makes it easy to learn non-linear, distributed
		  representations one layer at a time. The mul- tiple layers
		  of representation learned in this way can subsequently be
		  ﬁne-tuned to produce generative or discriminative models
		  that work much better than previous approaches.},
  added-at	= {2009-06-02T18:51:39.000+0200},
  author	= {Hinton, Geoffrey E.},
  biburl	= {http://www.bibsonomy.org/bibtex/2e3c887150ddeb929540f87bac5a26008/rcreswick}
		  ,
  citeulike-article-id={2599867},
  interhash	= {6b17388236cbabbcdd76f466b0ce22a3},
  intrahash	= {e3c887150ddeb929540f87bac5a26008},
  journal	= {Trends in Cognitive Sciences},
  keywords	= {learning},
  number	= {10},
  posted-at	= {2008-03-26 22:08:00},
  priority	= {2},
  title		= {Learning multiple layers of representation},
  volume	= {11},
  year		= {2007},
  url_pdf	= {absps/tics.pdf}
}

Downloads: 1