Striving for Simplicity: The All Convolutional Net

Striving for Simplicity: The All Convolutional Net. Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. arXiv:1412.6806 [cs], December, 2014. arXiv: 1412.6806

Paper abstract bibtex

Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We ﬁnd that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this ﬁnding – and building on other recent work for ﬁnding simple network structures – we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the “deconvolution approach” for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.

@article{springenberg_striving_2014,
	title = {Striving for {Simplicity}: {The} {All} {Convolutional} {Net}},
	shorttitle = {Striving for {Simplicity}},
	url = {http://arxiv.org/abs/1412.6806},
	abstract = {Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We ﬁnd that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this ﬁnding – and building on other recent work for ﬁnding simple network structures – we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the “deconvolution approach” for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.},
	language = {en},
	urldate = {2019-03-03},
	journal = {arXiv:1412.6806 [cs]},
	author = {Springenberg, Jost Tobias and Dosovitskiy, Alexey and Brox, Thomas and Riedmiller, Martin},
	month = dec,
	year = {2014},
	note = {arXiv: 1412.6806},
	keywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing},
}

Downloads: 0

{"_id":"uuTscBptfriM39cau","bibbaseid":"springenberg-dosovitskiy-brox-riedmiller-strivingforsimplicitytheallconvolutionalnet-2014","author_short":["Springenberg, J. T.","Dosovitskiy, A.","Brox, T.","Riedmiller, M."],"bibdata":{"bibtype":"article","type":"article","title":"Striving for Simplicity: The All Convolutional Net","shorttitle":"Striving for Simplicity","url":"http://arxiv.org/abs/1412.6806","abstract":"Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We ﬁnd that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this ﬁnding – and building on other recent work for ﬁnding simple network structures – we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the “deconvolution approach” for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.","language":"en","urldate":"2019-03-03","journal":"arXiv:1412.6806 [cs]","author":[{"propositions":[],"lastnames":["Springenberg"],"firstnames":["Jost","Tobias"],"suffixes":[]},{"propositions":[],"lastnames":["Dosovitskiy"],"firstnames":["Alexey"],"suffixes":[]},{"propositions":[],"lastnames":["Brox"],"firstnames":["Thomas"],"suffixes":[]},{"propositions":[],"lastnames":["Riedmiller"],"firstnames":["Martin"],"suffixes":[]}],"month":"December","year":"2014","note":"arXiv: 1412.6806","keywords":"Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing","bibtex":"@article{springenberg_striving_2014,\n\ttitle = {Striving for {Simplicity}: {The} {All} {Convolutional} {Net}},\n\tshorttitle = {Striving for {Simplicity}},\n\turl = {http://arxiv.org/abs/1412.6806},\n\tabstract = {Most modern convolutional neural networks (CNNs) used for object recognition are built using the same principles: Alternating convolution and max-pooling layers followed by a small number of fully connected layers. We re-evaluate the state of the art for object recognition from small images with convolutional networks, questioning the necessity of different components in the pipeline. We ﬁnd that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks. Following this ﬁnding – and building on other recent work for ﬁnding simple network structures – we propose a new architecture that consists solely of convolutional layers and yields competitive or state of the art performance on several object recognition datasets (CIFAR-10, CIFAR-100, ImageNet). To analyze the network we introduce a new variant of the “deconvolution approach” for visualizing features learned by CNNs, which can be applied to a broader range of network structures than existing approaches.},\n\tlanguage = {en},\n\turldate = {2019-03-03},\n\tjournal = {arXiv:1412.6806 [cs]},\n\tauthor = {Springenberg, Jost Tobias and Dosovitskiy, Alexey and Brox, Thomas and Riedmiller, Martin},\n\tmonth = dec,\n\tyear = {2014},\n\tnote = {arXiv: 1412.6806},\n\tkeywords = {Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Neural and Evolutionary Computing},\n}\n\n","author_short":["Springenberg, J. T.","Dosovitskiy, A.","Brox, T.","Riedmiller, M."],"key":"springenberg_striving_2014","id":"springenberg_striving_2014","bibbaseid":"springenberg-dosovitskiy-brox-riedmiller-strivingforsimplicitytheallconvolutionalnet-2014","role":"author","urls":{"Paper":"http://arxiv.org/abs/1412.6806"},"keyword":["Computer Science - Computer Vision and Pattern Recognition","Computer Science - Machine Learning","Computer Science - Neural and Evolutionary Computing"],"metadata":{"authorlinks":{}},"html":""},"bibtype":"article","biburl":"https://bibbase.org/zotero/sede","dataSources":["pQYee5oovEhJF75n6"],"keywords":["computer science - computer vision and pattern recognition","computer science - machine learning","computer science - neural and evolutionary computing"],"search_terms":["striving","simplicity","convolutional","net","springenberg","dosovitskiy","brox","riedmiller"],"title":"Striving for Simplicity: The All Convolutional Net","year":2014}