Investigating associative, switchable and negatable Winograd items on renewed French data sets. Wang, X., Seminck, O., & Amsili, P. In Actes de la 29e Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 : conférence principale, pages 136–143, Avignon, France, 6, 2022. ATALA. ![Investigating associative, switchable and negatable Winograd items on renewed French data sets [pdf]](https://bibbase.org/img/filetypes/pdf.svg "pdf")
Pdf Slides ![Investigating associative, switchable and negatable Winograd items on renewed French data sets [link]](https://bibbase.org/img/filetypes/link.svg "link")
Acl Hal abstract bibtex 4 downloads The Winograd Schema Challenge (WSC) consists of a set of anaphora resolution problems resolvable only by reasoning about world knowledge. This article describes the update of the existing French data set and the creation of three subsets allowing for a more robust, fine-grained evaluation protocol of WSC in French (FWSC) : an associative subset (items easily resolvable with lexical co-occurrence), a switchable subset (items where the inversion of two keywords reverses the answer) and a negatable subset (items where applying negation on its verb reverses the answer). Experiences on these data sets with CamemBERT reach SOTA performances. Our evaluation protocol showed in addition that the higher performance could be explained by the existence of associative items in FWSC. Besides, increasing the size of training corpus improves the model's performance on switchable items while the impact of larger training corpus remains small on negatable items.
@InProceedings{ wang22.taln,
title = "Investigating associative, switchable and negatable
{W}inograd items on renewed {F}rench data sets",
author = "Wang, Xiaoou and Seminck, Olga and Amsili, Pascal",
booktitle = "Actes de la 29e Conf{\'e}rence sur le Traitement
Automatique des Langues Naturelles. Volume 1 :
conf{\'e}rence principale",
month = 6,
year = 2022,
address = "Avignon, France",
publisher = "ATALA",
pages = "136--143",
abstract = "The Winograd Schema Challenge (WSC) consists of a set of
anaphora resolution problems resolvable only by reasoning
about world knowledge. This article describes the update of
the existing French data set and the creation of three
subsets allowing for a more robust, fine-grained evaluation
protocol of WSC in French (FWSC) : an associative subset
(items easily resolvable with lexical co-occurrence), a
switchable subset (items where the inversion of two
keywords reverses the answer) and a negatable subset (items
where applying negation on its verb reverses the answer).
Experiences on these data sets with CamemBERT reach SOTA
performances. Our evaluation protocol showed in addition
that the higher performance could be explained by the
existence of associative items in FWSC. Besides, increasing
the size of training corpus improves the model{'}s
performance on switchable items while the impact of larger
training corpus remains small on negatable items.",
url_pdf = {../Docs/papers/wang22_taln.pdf},
url_slides = {../Docs/talks/slides_wang22_taln.pdf},
url_acl = "https://aclanthology.org/2022.jeptalnrecital-taln.13",
url_hal = {https://hal.science/hal-03701511v1}
}
Downloads: 4
{"_id":"x6EQQg2sAjWZTgLnN","bibbaseid":"wang-seminck-amsili-investigatingassociativeswitchableandnegatablewinograditemsonrenewedfrenchdatasets-2022","author_short":["Wang, X.","Seminck, O.","Amsili, P."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"Investigating associative, switchable and negatable Winograd items on renewed French data sets","author":[{"propositions":[],"lastnames":["Wang"],"firstnames":["Xiaoou"],"suffixes":[]},{"propositions":[],"lastnames":["Seminck"],"firstnames":["Olga"],"suffixes":[]},{"propositions":[],"lastnames":["Amsili"],"firstnames":["Pascal"],"suffixes":[]}],"booktitle":"Actes de la 29e Conférence sur le Traitement Automatique des Langues Naturelles. Volume 1 : conférence principale","month":"6","year":"2022","address":"Avignon, France","publisher":"ATALA","pages":"136–143","abstract":"The Winograd Schema Challenge (WSC) consists of a set of anaphora resolution problems resolvable only by reasoning about world knowledge. This article describes the update of the existing French data set and the creation of three subsets allowing for a more robust, fine-grained evaluation protocol of WSC in French (FWSC) : an associative subset (items easily resolvable with lexical co-occurrence), a switchable subset (items where the inversion of two keywords reverses the answer) and a negatable subset (items where applying negation on its verb reverses the answer). Experiences on these data sets with CamemBERT reach SOTA performances. Our evaluation protocol showed in addition that the higher performance could be explained by the existence of associative items in FWSC. Besides, increasing the size of training corpus improves the model's performance on switchable items while the impact of larger training corpus remains small on negatable items.","url_pdf":"../Docs/papers/wang22_taln.pdf","url_slides":"../Docs/talks/slides_wang22_taln.pdf","url_acl":"https://aclanthology.org/2022.jeptalnrecital-taln.13","url_hal":"https://hal.science/hal-03701511v1","bibtex":"@InProceedings{\t wang22.taln,\n title\t\t= \"Investigating associative, switchable and negatable\n\t\t {W}inograd items on renewed {F}rench data sets\",\n author\t= \"Wang, Xiaoou and Seminck, Olga and Amsili, Pascal\",\n booktitle\t= \"Actes de la 29e Conf{\\'e}rence sur le Traitement\n\t\t Automatique des Langues Naturelles. Volume 1 :\n\t\t conf{\\'e}rence principale\",\n month\t\t= 6,\n year\t\t= 2022,\n address\t= \"Avignon, France\",\n publisher\t= \"ATALA\",\n pages\t\t= \"136--143\",\n abstract\t= \"The Winograd Schema Challenge (WSC) consists of a set of\n\t\t anaphora resolution problems resolvable only by reasoning\n\t\t about world knowledge. This article describes the update of\n\t\t the existing French data set and the creation of three\n\t\t subsets allowing for a more robust, fine-grained evaluation\n\t\t protocol of WSC in French (FWSC) : an associative subset\n\t\t (items easily resolvable with lexical co-occurrence), a\n\t\t switchable subset (items where the inversion of two\n\t\t keywords reverses the answer) and a negatable subset (items\n\t\t where applying negation on its verb reverses the answer).\n\t\t Experiences on these data sets with CamemBERT reach SOTA\n\t\t performances. Our evaluation protocol showed in addition\n\t\t that the higher performance could be explained by the\n\t\t existence of associative items in FWSC. Besides, increasing\n\t\t the size of training corpus improves the model{'}s\n\t\t performance on switchable items while the impact of larger\n\t\t training corpus remains small on negatable items.\",\n url_pdf\t= {../Docs/papers/wang22_taln.pdf},\n url_slides\t= {../Docs/talks/slides_wang22_taln.pdf},\n url_acl\t= \"https://aclanthology.org/2022.jeptalnrecital-taln.13\",\n url_hal\t= {https://hal.science/hal-03701511v1}\n}\n\n","author_short":["Wang, X.","Seminck, O.","Amsili, P."],"key":"wang22.taln","id":"wang22.taln","bibbaseid":"wang-seminck-amsili-investigatingassociativeswitchableandnegatablewinograditemsonrenewedfrenchdatasets-2022","role":"author","urls":{" pdf":"http://www.linguist.univ-paris-diderot.fr/~amsili/Docs/papers/wang22_taln.pdf"," slides":"http://www.linguist.univ-paris-diderot.fr/~amsili/Docs/talks/slides_wang22_taln.pdf"," acl":"https://aclanthology.org/2022.jeptalnrecital-taln.13"," hal":"https://hal.science/hal-03701511v1"},"metadata":{"authorlinks":{}},"downloads":4,"html":""},"bibtype":"inproceedings","biburl":"http://www.linguist.univ-paris-diderot.fr/~amsili/Rech/amsili.bib","dataSources":["iNFr6gDFmyXDAnd9E","G8v5dYGnBDTY4xtwo"],"keywords":[],"search_terms":["investigating","associative","switchable","negatable","winograd","items","renewed","french","data","sets","wang","seminck","amsili"],"title":"Investigating associative, switchable and negatable Winograd items on renewed French data sets","year":2022,"downloads":4}