Learning to represent visual input

Learning to represent visual input. Hinton, G. E. Philosophical Transactions of the Royal Society B: Biological Sciences, 365(1537):177–184, January, 2010.

Link

Learning to represent visual input [pdf]

Pdf doi abstract bibtex

10.1098/rstb.2009.0200 One of the central problems in computational neuroscience is to understand how the object-recognition pathway of the cortex learns a deep hierarchy of nonlinear feature detectors. Recent progress in machine learning shows that it is possible to learn deep hierarchies without requiring any labelled data. The feature detectors are learned one layer at a time and the goal of the learning procedure is to form a good generative model of images, not to predict the class of each image. The learning procedure only requires the pairwise correlations between the activations of neuron-like processing units in adjacent layers. The original version of the learning procedure is derived from a quadratic âenergyâ function but it can be extended to allow third-order, multiplicative interactions in which neurons gate the pairwise interactions between other neurons. A technique for factoring the third-order interactions leads to a learning module that again has a simple learning rule based on pairwise correlations. This module looks remarkably like modules that have been proposed by both biologists trying to explain the responses of neurons and engineers trying to create systems that can recognize objects.

@Article{	  citeulike:6377969,
  abstract	= {{10.1098/rstb.2009.0200 One of the central problems in
		  computational neuroscience is to understand how the
		  object-recognition pathway of the cortex learns a deep
		  hierarchy of nonlinear feature detectors. Recent progress
		  in machine learning shows that it is possible to learn deep
		  hierarchies without requiring any labelled data. The
		  feature detectors are learned one layer at a time and the
		  goal of the learning procedure is to form a good generative
		  model of images, not to predict the class of each image.
		  The learning procedure only requires the pairwise
		  correlations between the activations of neuron-like
		  processing units in adjacent layers. The original version
		  of the learning procedure is derived from a quadratic
		  âenergyâ function but it can be extended to allow
		  third-order, multiplicative interactions in which neurons
		  gate the pairwise interactions between other neurons. A
		  technique for factoring the third-order interactions leads
		  to a learning module that again has a simple learning rule
		  based on pairwise correlations. This module looks
		  remarkably like modules that have been proposed by both
		  biologists trying to explain the responses of neurons and
		  engineers trying to create systems that can recognize
		  objects.}},
  added-at	= {2010-11-30T22:39:03.000+0100},
  author	= {Hinton, Geoffrey E.},
  biburl	= {http://www.bibsonomy.org/bibtex/2d6b206f4e844cddf8c2350da9a05edcb/smatthiesen}
		  ,
  citeulike-article-id={6377969},
  citeulike-linkout-0={http://dx.doi.org/10.1098/rstb.2009.0200},
  citeulike-linkout-1={http://rstb.royalsocietypublishing.org/content/365/1537/177.abstract}
		  ,
  citeulike-linkout-2={http://rstb.royalsocietypublishing.org/content/365/1537/177.full.pdf}
		  ,
  day		= {12},
  doi		= {10.1098/rstb.2009.0200},
  interhash	= {8202ab76a45c599dec3f83ecf63e5534},
  intrahash	= {d6b206f4e844cddf8c2350da9a05edcb},
  journal	= {Philosophical Transactions of the Royal Society B:
		  Biological Sciences},
  keywords	= {object recognition},
  month		= {January},
  number	= {1537},
  pages		= {177--184},
  posted-at	= {2009-12-15 08:37:24},
  priority	= {2},
  title		= {{Learning to represent visual input}},
  url_link	= {http://dx.doi.org/10.1098/rstb.2009.0200},
  volume	= {365},
  year		= {2010},
  url_pdf	= {absps/Philtrans2010.pdf}
}

Downloads: 0