Minimum Description Length Hopfield Networks. Abudy, M., Lan, N., Chemla, E., & Katzir, R. In Special Collection on Associative Memory and Hopfield Networks in PRX Life, 2023. To appear![Minimum Description Length Hopfield Networks [link]](https://bibbase.org/img/filetypes/link.svg "link")
More abstract bibtex Associative memory architectures are designed for memorization but also offer, through their retrieval method, a form of generalization to unseen inputs: stored memories can be seen as prototypes from this point of view. Focusing on Modern Hopfield Networks (MHN), we show that a large memorization capacity undermines the generalization opportunity. We offer a solution to better optimize this tradeoff. It relies on Minimum Description Length (MDL) to determine during training which memories to store, as well as how many of them.
@inproceedings{MDLHN,
abstract = {Associative memory architectures are designed for memorization but also offer, through their retrieval method, a form of generalization to unseen inputs: stored memories can be seen as prototypes from this point of view. Focusing on Modern Hopfield Networks (MHN), we show that a large memorization capacity undermines the generalization opportunity. We offer a solution to better optimize this tradeoff. It relies on Minimum Description Length (MDL) to determine during training which memories to store, as well as how many of them.},
author = {Matan Abudy and Nur Lan and Emmanuel Chemla and Roni Katzir},
booktitle = {Special Collection on Associative Memory and Hopfield Networks in PRX Life},
date-added = {2023-12-20 15:46:34 +0100},
date-modified = {2023-12-20 15:48:43 +0100},
note = {To appear},
title = {Minimum Description Length Hopfield Networks},
url_more = {https://arxiv.org/abs/2311.06518},
year = {2023}}
Downloads: 0
{"_id":"EcKQcafLa45e6QnEM","bibbaseid":"abudy-lan-chemla-katzir-minimumdescriptionlengthhopfieldnetworks-2023","author_short":["Abudy, M.","Lan, N.","Chemla, E.","Katzir, R."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","abstract":"Associative memory architectures are designed for memorization but also offer, through their retrieval method, a form of generalization to unseen inputs: stored memories can be seen as prototypes from this point of view. Focusing on Modern Hopfield Networks (MHN), we show that a large memorization capacity undermines the generalization opportunity. We offer a solution to better optimize this tradeoff. It relies on Minimum Description Length (MDL) to determine during training which memories to store, as well as how many of them.","author":[{"firstnames":["Matan"],"propositions":[],"lastnames":["Abudy"],"suffixes":[]},{"firstnames":["Nur"],"propositions":[],"lastnames":["Lan"],"suffixes":[]},{"firstnames":["Emmanuel"],"propositions":[],"lastnames":["Chemla"],"suffixes":[]},{"firstnames":["Roni"],"propositions":[],"lastnames":["Katzir"],"suffixes":[]}],"booktitle":"Special Collection on Associative Memory and Hopfield Networks in PRX Life","date-added":"2023-12-20 15:46:34 +0100","date-modified":"2023-12-20 15:48:43 +0100","note":"To appear","title":"Minimum Description Length Hopfield Networks","url_more":"https://arxiv.org/abs/2311.06518","year":"2023","bibtex":"@inproceedings{MDLHN,\n\tabstract = {Associative memory architectures are designed for memorization but also offer, through their retrieval method, a form of generalization to unseen inputs: stored memories can be seen as prototypes from this point of view. Focusing on Modern Hopfield Networks (MHN), we show that a large memorization capacity undermines the generalization opportunity. We offer a solution to better optimize this tradeoff. It relies on Minimum Description Length (MDL) to determine during training which memories to store, as well as how many of them.},\n\tauthor = {Matan Abudy and Nur Lan and Emmanuel Chemla and Roni Katzir},\n\tbooktitle = {Special Collection on Associative Memory and Hopfield Networks in PRX Life},\n\tdate-added = {2023-12-20 15:46:34 +0100},\n\tdate-modified = {2023-12-20 15:48:43 +0100},\n\tnote = {To appear},\n\ttitle = {Minimum Description Length Hopfield Networks},\n\turl_more = {https://arxiv.org/abs/2311.06518},\n\tyear = {2023}}\n\n","author_short":["Abudy, M.","Lan, N.","Chemla, E.","Katzir, R."],"key":"MDLHN","id":"MDLHN","bibbaseid":"abudy-lan-chemla-katzir-minimumdescriptionlengthhopfieldnetworks-2023","role":"author","urls":{" more":"https://arxiv.org/abs/2311.06518"},"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"http://www.emmanuel.chemla.free.fr/Material/EmmanuelChemlaBibliography.bib","dataSources":["tjiGpM3QkXrAR3DJb"],"keywords":[],"search_terms":["minimum","description","length","hopfield","networks","abudy","lan","chemla","katzir"],"title":"Minimum Description Length Hopfield Networks","year":2023}