Learning fast approximations of sparse coding. Gregor, K. & LeCun, Y. In Proceedings of the 27th International Conference on International Conference on Machine Learning, of ICML'10, pages 399–406, Madison, WI, USA, June, 2010. Omnipress. abstract bibtex In Sparse Coding (SC), input vectors are reconstructed using a sparse linear combination of basis vectors. SC has become a popular method for extracting features from data. For a given input, SC minimizes a quadratic reconstruction error with an L1 penalty term on the code. The process is often too slow for applications such as real-time pattern recognition. We proposed two versions of a very fast algorithm that produces approximate estimates of the sparse code that can be used to compute good visual features, or to initialize exact iterative algorithms. The main idea is to train a non-linear, feed-forward predictor with a specific architecture and a fixed depth to produce the best possible approximation of the sparse code. A version of the method, which can be seen as a trainable version of Li and Osher's coordinate descent method, is shown to produce approximate solutions with 10 times less computation than Li and Os-her's for the same approximation error. Unlike previous proposals for sparse code predictors, the system allows a kind of approximate "explaining away" to take place during inference. The resulting predictor is differentiable and can be included into globally-trained recognition systems.
@inproceedings{gregor_learning_2010,
address = {Madison, WI, USA},
series = {{ICML}'10},
title = {Learning fast approximations of sparse coding},
isbn = {978-1-60558-907-7},
abstract = {In Sparse Coding (SC), input vectors are reconstructed using a sparse linear combination of basis vectors. SC has become a popular method for extracting features from data. For a given input, SC minimizes a quadratic reconstruction error with an L1 penalty term on the code. The process is often too slow for applications such as real-time pattern recognition. We proposed two versions of a very fast algorithm that produces approximate estimates of the sparse code that can be used to compute good visual features, or to initialize exact iterative algorithms. The main idea is to train a non-linear, feed-forward predictor with a specific architecture and a fixed depth to produce the best possible approximation of the sparse code. A version of the method, which can be seen as a trainable version of Li and Osher's coordinate descent method, is shown to produce approximate solutions with 10 times less computation than Li and Os-her's for the same approximation error. Unlike previous proposals for sparse code predictors, the system allows a kind of approximate "explaining away" to take place during inference. The resulting predictor is differentiable and can be included into globally-trained recognition systems.},
language = {en},
urldate = {2023-07-03},
booktitle = {Proceedings of the 27th {International} {Conference} on {International} {Conference} on {Machine} {Learning}},
publisher = {Omnipress},
author = {Gregor, Karol and LeCun, Yann},
month = jun,
year = {2010},
keywords = {\#Analysis, \#ICML{\textgreater}10, \#Sparse, /unread, ⭐⭐⭐⭐⭐, 💡},
pages = {399--406},
}
Downloads: 0
{"_id":"7goLpNeuq5G7J7bdh","bibbaseid":"gregor-lecun-learningfastapproximationsofsparsecoding-2010","authorIDs":[],"author_short":["Gregor, K.","LeCun, Y."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Madison, WI, USA","series":"ICML'10","title":"Learning fast approximations of sparse coding","isbn":"978-1-60558-907-7","abstract":"In Sparse Coding (SC), input vectors are reconstructed using a sparse linear combination of basis vectors. SC has become a popular method for extracting features from data. For a given input, SC minimizes a quadratic reconstruction error with an L1 penalty term on the code. The process is often too slow for applications such as real-time pattern recognition. We proposed two versions of a very fast algorithm that produces approximate estimates of the sparse code that can be used to compute good visual features, or to initialize exact iterative algorithms. The main idea is to train a non-linear, feed-forward predictor with a specific architecture and a fixed depth to produce the best possible approximation of the sparse code. A version of the method, which can be seen as a trainable version of Li and Osher's coordinate descent method, is shown to produce approximate solutions with 10 times less computation than Li and Os-her's for the same approximation error. Unlike previous proposals for sparse code predictors, the system allows a kind of approximate \"explaining away\" to take place during inference. The resulting predictor is differentiable and can be included into globally-trained recognition systems.","language":"en","urldate":"2023-07-03","booktitle":"Proceedings of the 27th International Conference on International Conference on Machine Learning","publisher":"Omnipress","author":[{"propositions":[],"lastnames":["Gregor"],"firstnames":["Karol"],"suffixes":[]},{"propositions":[],"lastnames":["LeCun"],"firstnames":["Yann"],"suffixes":[]}],"month":"June","year":"2010","keywords":"#Analysis, #ICML\\textgreater10, #Sparse, /unread, ⭐⭐⭐⭐⭐, 💡","pages":"399–406","bibtex":"@inproceedings{gregor_learning_2010,\n\taddress = {Madison, WI, USA},\n\tseries = {{ICML}'10},\n\ttitle = {Learning fast approximations of sparse coding},\n\tisbn = {978-1-60558-907-7},\n\tabstract = {In Sparse Coding (SC), input vectors are reconstructed using a sparse linear combination of basis vectors. SC has become a popular method for extracting features from data. For a given input, SC minimizes a quadratic reconstruction error with an L1 penalty term on the code. The process is often too slow for applications such as real-time pattern recognition. We proposed two versions of a very fast algorithm that produces approximate estimates of the sparse code that can be used to compute good visual features, or to initialize exact iterative algorithms. The main idea is to train a non-linear, feed-forward predictor with a specific architecture and a fixed depth to produce the best possible approximation of the sparse code. A version of the method, which can be seen as a trainable version of Li and Osher's coordinate descent method, is shown to produce approximate solutions with 10 times less computation than Li and Os-her's for the same approximation error. Unlike previous proposals for sparse code predictors, the system allows a kind of approximate \"explaining away\" to take place during inference. The resulting predictor is differentiable and can be included into globally-trained recognition systems.},\n\tlanguage = {en},\n\turldate = {2023-07-03},\n\tbooktitle = {Proceedings of the 27th {International} {Conference} on {International} {Conference} on {Machine} {Learning}},\n\tpublisher = {Omnipress},\n\tauthor = {Gregor, Karol and LeCun, Yann},\n\tmonth = jun,\n\tyear = {2010},\n\tkeywords = {\\#Analysis, \\#ICML{\\textgreater}10, \\#Sparse, /unread, ⭐⭐⭐⭐⭐, 💡},\n\tpages = {399--406},\n}\n\n\n\n","author_short":["Gregor, K.","LeCun, Y."],"key":"gregor_learning_2010","id":"gregor_learning_2010","bibbaseid":"gregor-lecun-learningfastapproximationsofsparsecoding-2010","role":"author","urls":{},"keyword":["#Analysis","#ICML\\textgreater10","#Sparse","/unread","⭐⭐⭐⭐⭐","💡"],"metadata":{"authorlinks":{}},"downloads":0,"html":""},"bibtype":"inproceedings","biburl":"https://bibbase.org/zotero/zzhenry2012","creationDate":"2020-01-27T02:13:34.446Z","downloads":0,"keywords":["#analysis","#icml\\textgreater10","#sparse","/unread","⭐⭐⭐⭐⭐","💡"],"search_terms":["learning","fast","approximations","sparse","coding","gregor","lecun"],"title":"Learning fast approximations of sparse coding","year":2010,"dataSources":["hEoKh4ygEAWbAZ5iy","nZHrFJKyxKKDaWYM8"]}