PaCo: Preconditions Attributed to Commonsense Knowledge. Qasemi, E., Ilievski, F., Chen, M., & Szekely, P. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing-Findings, pages 6781–6796, Abu Dhabi, United Arab Emirates, December, 2022. Association for Computational Linguistics. ![PaCo: Preconditions Attributed to Commonsense Knowledge [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that a glass is used for drinking water, unless the glass is broken or the water is toxic. Despite state-of-the-art (SOTA) language models’ (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge.
@inproceedings{qasemi2022paco,
title = "PaCo: Preconditions Attributed to Commonsense Knowledge",
author = "Qasemi, Ehsan and Ilievski, Filip and Chen, Muhao and Szekely, Pedro",
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing-Findings",
month = dec,
year = "2022",
address = "Abu Dhabi, United Arab Emirates",
publisher = "Association for Computational Linguistics",
url = "https://preview.aclanthology.org/emnlp-22-ingestion/2022.findings-emnlp.505/",
doi = "10.48550/ARXIV.2104.08712",
pages = "6781–6796",
abstract = "Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that a glass is used for drinking water, unless the glass is broken or the water is toxic. Despite state-of-the-art (SOTA) language models’ (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30\% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge.",
ISIArea = {NLP}
}
Downloads: 0
{"_id":"m5DbzwCbAoaij73rG","bibbaseid":"qasemi-ilievski-chen-szekely-pacopreconditionsattributedtocommonsenseknowledge-2022","author_short":["Qasemi, E.","Ilievski, F.","Chen, M.","Szekely, P."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"PaCo: Preconditions Attributed to Commonsense Knowledge","author":[{"propositions":[],"lastnames":["Qasemi"],"firstnames":["Ehsan"],"suffixes":[]},{"propositions":[],"lastnames":["Ilievski"],"firstnames":["Filip"],"suffixes":[]},{"propositions":[],"lastnames":["Chen"],"firstnames":["Muhao"],"suffixes":[]},{"propositions":[],"lastnames":["Szekely"],"firstnames":["Pedro"],"suffixes":[]}],"booktitle":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing-Findings","month":"December","year":"2022","address":"Abu Dhabi, United Arab Emirates","publisher":"Association for Computational Linguistics","url":"https://preview.aclanthology.org/emnlp-22-ingestion/2022.findings-emnlp.505/","doi":"10.48550/ARXIV.2104.08712","pages":"6781–6796","abstract":"Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that a glass is used for drinking water, unless the glass is broken or the water is toxic. Despite state-of-the-art (SOTA) language models’ (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge.","isiarea":"NLP","bibtex":"@inproceedings{qasemi2022paco,\n title = \"PaCo: Preconditions Attributed to Commonsense Knowledge\",\n author = \"Qasemi, Ehsan and Ilievski, Filip and Chen, Muhao and Szekely, Pedro\",\n booktitle = \"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing-Findings\",\n month = dec,\n year = \"2022\",\n address = \"Abu Dhabi, United Arab Emirates\",\n publisher = \"Association for Computational Linguistics\",\n url = \"https://preview.aclanthology.org/emnlp-22-ingestion/2022.findings-emnlp.505/\",\n doi = \"10.48550/ARXIV.2104.08712\",\n pages = \"6781–6796\",\n abstract = \"Humans can seamlessly reason with circumstantial preconditions of commonsense knowledge. We understand that a glass is used for drinking water, unless the glass is broken or the water is toxic. Despite state-of-the-art (SOTA) language models’ (LMs) impressive performance on inferring commonsense knowledge, it is unclear whether they understand the circumstantial preconditions. To address this gap, we propose a novel challenge of reasoning with circumstantial preconditions. We collect a dataset, called PaCo, consisting of 12.4 thousand preconditions of commonsense statements expressed in natural language. Based on this dataset, we create three canonical evaluation tasks and use them to examine the capability of existing LMs to understand situational preconditions. Our results reveal a 10-30\\% gap between machine and human performance on our tasks, which shows that reasoning with preconditions is an open challenge.\",\n ISIArea = {NLP} \n}\n","author_short":["Qasemi, E.","Ilievski, F.","Chen, M.","Szekely, P."],"bibbaseid":"qasemi-ilievski-chen-szekely-pacopreconditionsattributedtocommonsenseknowledge-2022","role":"author","urls":{"Paper":"https://preview.aclanthology.org/emnlp-22-ingestion/2022.findings-emnlp.505/"},"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://bibbase.org/f/EKqKhGdsyDWEBTkE8/qasemi.bib","dataSources":["u4jXiFwYhKs7oMpKf","9X3Rw7XQYB2MtTnNq"],"keywords":[],"search_terms":["paco","preconditions","attributed","commonsense","knowledge","qasemi","ilievski","chen","szekely"],"title":"PaCo: Preconditions Attributed to Commonsense Knowledge","year":2022}