Stochastic Gradient Descent Tricks

Stochastic Gradient Descent Tricks. Bottou, L. In Montavon, G., Orr, G. B., & Müller, K., editors, Neural Networks: Tricks of the Trade: Second Edition, of Lecture Notes in Computer Science, pages 421–436. Springer, Berlin, Heidelberg, 2012.

Paper doi abstract bibtex

Chapter 1 strongly advocates the stochastic back-propagation method to train neural networks. This is in fact an instance of a more general technique called stochastic gradient descent (SGD). This chapter provides background material, explains why SGD is a good learning algorithm when the training set is large, and provides useful recommendations.

@incollection{bottou_stochastic_2012,
	address = {Berlin, Heidelberg},
	series = {Lecture {Notes} in {Computer} {Science}},
	title = {Stochastic {Gradient} {Descent} {Tricks}},
	isbn = {978-3-642-35289-8},
	url = {https://doi.org/10.1007/978-3-642-35289-8_25},
	abstract = {Chapter 1 strongly advocates the stochastic back-propagation method to train neural networks. This is in fact an instance of a more general technique called stochastic gradient descent (SGD). This chapter provides background material, explains why SGD is a good learning algorithm when the training set is large, and provides useful recommendations.},
	language = {en},
	urldate = {2022-03-19},
	booktitle = {Neural {Networks}: {Tricks} of the {Trade}: {Second} {Edition}},
	publisher = {Springer},
	author = {Bottou, Léon},
	editor = {Montavon, Grégoire and Orr, Geneviève B. and Müller, Klaus-Robert},
	year = {2012},
	doi = {10.1007/978-3-642-35289-8_25},
	keywords = {Conditional Random Field, Empirical Risk, Learning Rate, Stochastic Gradient, Support Vector Machine},
	pages = {421--436},
}

Downloads: 0

{"_id":"sj8XFJYA7RGPkXHyM","bibbaseid":"bottou-stochasticgradientdescenttricks-2012","authorIDs":[],"author_short":["Bottou, L."],"bibdata":{"bibtype":"incollection","type":"incollection","address":"Berlin, Heidelberg","series":"Lecture Notes in Computer Science","title":"Stochastic Gradient Descent Tricks","isbn":"978-3-642-35289-8","url":"https://doi.org/10.1007/978-3-642-35289-8_25","abstract":"Chapter 1 strongly advocates the stochastic back-propagation method to train neural networks. This is in fact an instance of a more general technique called stochastic gradient descent (SGD). This chapter provides background material, explains why SGD is a good learning algorithm when the training set is large, and provides useful recommendations.","language":"en","urldate":"2022-03-19","booktitle":"Neural Networks: Tricks of the Trade: Second Edition","publisher":"Springer","author":[{"propositions":[],"lastnames":["Bottou"],"firstnames":["Léon"],"suffixes":[]}],"editor":[{"propositions":[],"lastnames":["Montavon"],"firstnames":["Grégoire"],"suffixes":[]},{"propositions":[],"lastnames":["Orr"],"firstnames":["Geneviève","B."],"suffixes":[]},{"propositions":[],"lastnames":["Müller"],"firstnames":["Klaus-Robert"],"suffixes":[]}],"year":"2012","doi":"10.1007/978-3-642-35289-8_25","keywords":"Conditional Random Field, Empirical Risk, Learning Rate, Stochastic Gradient, Support Vector Machine","pages":"421–436","bibtex":"@incollection{bottou_stochastic_2012,\n\taddress = {Berlin, Heidelberg},\n\tseries = {Lecture {Notes} in {Computer} {Science}},\n\ttitle = {Stochastic {Gradient} {Descent} {Tricks}},\n\tisbn = {978-3-642-35289-8},\n\turl = {https://doi.org/10.1007/978-3-642-35289-8_25},\n\tabstract = {Chapter 1 strongly advocates the stochastic back-propagation method to train neural networks. This is in fact an instance of a more general technique called stochastic gradient descent (SGD). This chapter provides background material, explains why SGD is a good learning algorithm when the training set is large, and provides useful recommendations.},\n\tlanguage = {en},\n\turldate = {2022-03-19},\n\tbooktitle = {Neural {Networks}: {Tricks} of the {Trade}: {Second} {Edition}},\n\tpublisher = {Springer},\n\tauthor = {Bottou, Léon},\n\teditor = {Montavon, Grégoire and Orr, Geneviève B. and Müller, Klaus-Robert},\n\tyear = {2012},\n\tdoi = {10.1007/978-3-642-35289-8_25},\n\tkeywords = {Conditional Random Field, Empirical Risk, Learning Rate, Stochastic Gradient, Support Vector Machine},\n\tpages = {421--436},\n}\n\n","author_short":["Bottou, L."],"editor_short":["Montavon, G.","Orr, G. B.","Müller, K."],"key":"bottou_stochastic_2012","id":"bottou_stochastic_2012","bibbaseid":"bottou-stochasticgradientdescenttricks-2012","role":"author","urls":{"Paper":"https://doi.org/10.1007/978-3-642-35289-8_25"},"keyword":["Conditional Random Field","Empirical Risk","Learning Rate","Stochastic Gradient","Support Vector Machine"],"metadata":{"authorlinks":{}}},"bibtype":"incollection","biburl":"https://bibbase.org/zotero/mh_lenguyen","creationDate":"2020-01-27T02:13:34.326Z","downloads":0,"keywords":["conditional random field","empirical risk","learning rate","stochastic gradient","support vector machine"],"search_terms":["stochastic","gradient","descent","tricks","bottou"],"title":"Stochastic Gradient Descent Tricks","year":2012,"dataSources":["hEoKh4ygEAWbAZ5iy","iwKepCrWBps7ojhDx"]}