Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks. Lake, B. & Baroni, M. In Proceedings of the 35th International Conference on Machine Learning, pages 2873–2882, July, 2018. PMLR.
Paper abstract bibtex 10 downloads Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb "dax," he or she can immediately understand the meaning of "dax twice" or "sing and dax." In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply "mix-and-match" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the "dax" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.
@inproceedings{lakeGeneralizationSystematicityCompositional2018,
title = {Generalization without {{Systematicity}}: {{On}} the {{Compositional Skills}} of {{Sequence-to-Sequence Recurrent Networks}}},
shorttitle = {Generalization without {{Systematicity}}},
booktitle = {Proceedings of the 35th {{International Conference}} on {{Machine Learning}}},
author = {Lake, Brenden and Baroni, Marco},
year = {2018},
month = jul,
pages = {2873--2882},
publisher = {PMLR},
issn = {2640-3498},
url = {https://proceedings.mlr.press/v80/lake18a.html},
urldate = {2024-03-18},
abstract = {Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb "dax," he or she can immediately understand the meaning of "dax twice" or "sing and dax." In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply "mix-and-match" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the "dax" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.},
langid = {english},
file = {/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo2.pdf;/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo3.pdf}
}
Downloads: 10
{"_id":"vM9Xa4xQpnkCbzjiZ","bibbaseid":"lake-baroni-generalizationwithoutsystematicityonthecompositionalskillsofsequencetosequencerecurrentnetworks-2018","authorIDs":[],"author_short":["Lake, B.","Baroni, M."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks","shorttitle":"Generalization without Systematicity","booktitle":"Proceedings of the 35th International Conference on Machine Learning","author":[{"propositions":[],"lastnames":["Lake"],"firstnames":["Brenden"],"suffixes":[]},{"propositions":[],"lastnames":["Baroni"],"firstnames":["Marco"],"suffixes":[]}],"year":"2018","month":"July","pages":"2873–2882","publisher":"PMLR","issn":"2640-3498","url":"https://proceedings.mlr.press/v80/lake18a.html","urldate":"2024-03-18","abstract":"Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb \"dax,\" he or she can immediately understand the meaning of \"dax twice\" or \"sing and dax.\" In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply \"mix-and-match\" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the \"dax\" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.","langid":"english","file":"/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo2.pdf;/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo3.pdf","bibtex":"@inproceedings{lakeGeneralizationSystematicityCompositional2018,\n title = {Generalization without {{Systematicity}}: {{On}} the {{Compositional Skills}} of {{Sequence-to-Sequence Recurrent Networks}}},\n shorttitle = {Generalization without {{Systematicity}}},\n booktitle = {Proceedings of the 35th {{International Conference}} on {{Machine Learning}}},\n author = {Lake, Brenden and Baroni, Marco},\n year = {2018},\n month = jul,\n pages = {2873--2882},\n publisher = {PMLR},\n issn = {2640-3498},\n url = {https://proceedings.mlr.press/v80/lake18a.html},\n urldate = {2024-03-18},\n abstract = {Humans can understand and produce new utterances effortlessly, thanks to their compositional skills. Once a person learns the meaning of a new verb \"dax,\" he or she can immediately understand the meaning of \"dax twice\" or \"sing and dax.\" In this paper, we introduce the SCAN domain, consisting of a set of simple compositional navigation commands paired with the corresponding action sequences. We then test the zero-shot generalization capabilities of a variety of recurrent neural networks (RNNs) trained on SCAN with sequence-to-sequence methods. We find that RNNs can make successful zero-shot generalizations when the differences between training and test commands are small, so that they can apply \"mix-and-match\" strategies to solve the task. However, when generalization requires systematic compositional skills (as in the \"dax\" example above), RNNs fail spectacularly. We conclude with a proof-of-concept experiment in neural machine translation, suggesting that lack of systematicity might be partially responsible for neural networks' notorious training data thirst.},\n langid = {english},\n file = {/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo2.pdf;/Users/shanest/sync/library/Lake_Baroni/2018/Lake and Baroni - 2018 - Generalization without Systematicity On the Compo3.pdf}\n}\n\n","author_short":["Lake, B.","Baroni, M."],"key":"lakeGeneralizationSystematicityCompositional2018","id":"lakeGeneralizationSystematicityCompositional2018","bibbaseid":"lake-baroni-generalizationwithoutsystematicityonthecompositionalskillsofsequencetosequencerecurrentnetworks-2018","role":"author","urls":{"Paper":"https://proceedings.mlr.press/v80/lake18a.html"},"metadata":{"authorlinks":{}},"downloads":10},"bibtype":"inproceedings","biburl":"https://www.shane.st/teaching/575/spr24/575_Compositionality.bib","creationDate":"2020-01-05T04:04:02.892Z","downloads":10,"keywords":[],"search_terms":["generalization","without","systematicity","compositional","skills","sequence","sequence","recurrent","networks","lake","baroni"],"title":"Generalization without Systematicity: On the Compositional Skills of Sequence-to-Sequence Recurrent Networks","year":2018,"dataSources":["okYcdTpf4JJ2zkj7A","TPZs4iPqAgE5a8mjq","znj7izS5PeehdLR3G","aGtG992oMsrqA3Aas"]}