Expert Concept-Modeling Ground Truth Construction for Word Embeddings Evaluation in Concept-Focused Domains. Betti, A., Reynaert, M., Ossenkoppele, T., Oortwijn, Y., Salway, A., & Bloem, J. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6690–6702, Barcelona, Spain (Online), December, 2020. International Committee on Computational Linguistics. ![Expert Concept-Modeling Ground Truth Construction for Word Embeddings Evaluation in Concept-Focused Domains [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex 4 downloads We present a novel, domain expert-controlled, replicable procedure for the construction of concept-modeling ground truths with the aim of evaluating the application of word embeddings. In particular, our method is designed to evaluate the application of word and paragraph embeddings in concept-focused textual domains, where a generic ontology does not provide enough information. We illustrate the procedure, and validate it by describing the construction of an expert ground truth, QuiNE-GT. QuiNE-GT is built to answer research questions concerning the concept of naturalized epistemology in QUINE, a 2-million-token, single-author, 20th-century English philosophy corpus of outstanding quality, cleaned up and enriched for the purpose. To the best of our ken, expert concept-modeling ground truths are extremely rare in current literature, nor has the theoretical methodology behind their construction ever been explicitly conceptualised and properly systematised. Expert-controlled concept-modeling ground truths are however essential to allow proper evaluation of word embeddings techniques, and increase their trustworthiness in specialised domains in which the detection of concepts through their expression in texts is important. We highlight challenges, requirements, and prospects for future work.
@inproceedings{betti_expert_2020,
address = {Barcelona, Spain (Online)},
title = {Expert {Concept}-{Modeling} {Ground} {Truth} {Construction} for {Word} {Embeddings} {Evaluation} in {Concept}-{Focused} {Domains}},
url = {https://www.aclweb.org/anthology/2020.coling-main.586},
doi = {10.18653/v1/2020.coling-main.586},
abstract = {We present a novel, domain expert-controlled, replicable procedure for the construction of concept-modeling ground truths with the aim of evaluating the application of word embeddings. In particular, our method is designed to evaluate the application of word and paragraph embeddings in concept-focused textual domains, where a generic ontology does not provide enough information. We illustrate the procedure, and validate it by describing the construction of an expert ground truth, QuiNE-GT. QuiNE-GT is built to answer research questions concerning the concept of naturalized epistemology in QUINE, a 2-million-token, single-author, 20th-century English philosophy corpus of outstanding quality, cleaned up and enriched for the purpose. To the best of our ken, expert concept-modeling ground truths are extremely rare in current literature, nor has the theoretical methodology behind their construction ever been explicitly conceptualised and properly systematised. Expert-controlled concept-modeling ground truths are however essential to allow proper evaluation of word embeddings techniques, and increase their trustworthiness in specialised domains in which the detection of concepts through their expression in texts is important. We highlight challenges, requirements, and prospects for future work.},
urldate = {2021-01-26},
booktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}},
publisher = {International Committee on Computational Linguistics},
author = {Betti, Arianna and Reynaert, Martin and Ossenkoppele, Thijs and Oortwijn, Yvette and Salway, Andrew and Bloem, Jelke},
month = dec,
year = {2020},
pages = {6690--6702},
}
Downloads: 4
{"_id":"K9nXGJs4zWh8HQhxn","bibbaseid":"betti-reynaert-ossenkoppele-oortwijn-salway-bloem-expertconceptmodelinggroundtruthconstructionforwordembeddingsevaluationinconceptfocuseddomains-2020","authorIDs":["aeqmsSj8anitxzHwC"],"author_short":["Betti, A.","Reynaert, M.","Ossenkoppele, T.","Oortwijn, Y.","Salway, A.","Bloem, J."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Barcelona, Spain (Online)","title":"Expert Concept-Modeling Ground Truth Construction for Word Embeddings Evaluation in Concept-Focused Domains","url":"https://www.aclweb.org/anthology/2020.coling-main.586","doi":"10.18653/v1/2020.coling-main.586","abstract":"We present a novel, domain expert-controlled, replicable procedure for the construction of concept-modeling ground truths with the aim of evaluating the application of word embeddings. In particular, our method is designed to evaluate the application of word and paragraph embeddings in concept-focused textual domains, where a generic ontology does not provide enough information. We illustrate the procedure, and validate it by describing the construction of an expert ground truth, QuiNE-GT. QuiNE-GT is built to answer research questions concerning the concept of naturalized epistemology in QUINE, a 2-million-token, single-author, 20th-century English philosophy corpus of outstanding quality, cleaned up and enriched for the purpose. To the best of our ken, expert concept-modeling ground truths are extremely rare in current literature, nor has the theoretical methodology behind their construction ever been explicitly conceptualised and properly systematised. Expert-controlled concept-modeling ground truths are however essential to allow proper evaluation of word embeddings techniques, and increase their trustworthiness in specialised domains in which the detection of concepts through their expression in texts is important. We highlight challenges, requirements, and prospects for future work.","urldate":"2021-01-26","booktitle":"Proceedings of the 28th International Conference on Computational Linguistics","publisher":"International Committee on Computational Linguistics","author":[{"propositions":[],"lastnames":["Betti"],"firstnames":["Arianna"],"suffixes":[]},{"propositions":[],"lastnames":["Reynaert"],"firstnames":["Martin"],"suffixes":[]},{"propositions":[],"lastnames":["Ossenkoppele"],"firstnames":["Thijs"],"suffixes":[]},{"propositions":[],"lastnames":["Oortwijn"],"firstnames":["Yvette"],"suffixes":[]},{"propositions":[],"lastnames":["Salway"],"firstnames":["Andrew"],"suffixes":[]},{"propositions":[],"lastnames":["Bloem"],"firstnames":["Jelke"],"suffixes":[]}],"month":"December","year":"2020","pages":"6690–6702","bibtex":"@inproceedings{betti_expert_2020,\n\taddress = {Barcelona, Spain (Online)},\n\ttitle = {Expert {Concept}-{Modeling} {Ground} {Truth} {Construction} for {Word} {Embeddings} {Evaluation} in {Concept}-{Focused} {Domains}},\n\turl = {https://www.aclweb.org/anthology/2020.coling-main.586},\n\tdoi = {10.18653/v1/2020.coling-main.586},\n\tabstract = {We present a novel, domain expert-controlled, replicable procedure for the construction of concept-modeling ground truths with the aim of evaluating the application of word embeddings. In particular, our method is designed to evaluate the application of word and paragraph embeddings in concept-focused textual domains, where a generic ontology does not provide enough information. We illustrate the procedure, and validate it by describing the construction of an expert ground truth, QuiNE-GT. QuiNE-GT is built to answer research questions concerning the concept of naturalized epistemology in QUINE, a 2-million-token, single-author, 20th-century English philosophy corpus of outstanding quality, cleaned up and enriched for the purpose. To the best of our ken, expert concept-modeling ground truths are extremely rare in current literature, nor has the theoretical methodology behind their construction ever been explicitly conceptualised and properly systematised. Expert-controlled concept-modeling ground truths are however essential to allow proper evaluation of word embeddings techniques, and increase their trustworthiness in specialised domains in which the detection of concepts through their expression in texts is important. We highlight challenges, requirements, and prospects for future work.},\n\turldate = {2021-01-26},\n\tbooktitle = {Proceedings of the 28th {International} {Conference} on {Computational} {Linguistics}},\n\tpublisher = {International Committee on Computational Linguistics},\n\tauthor = {Betti, Arianna and Reynaert, Martin and Ossenkoppele, Thijs and Oortwijn, Yvette and Salway, Andrew and Bloem, Jelke},\n\tmonth = dec,\n\tyear = {2020},\n\tpages = {6690--6702},\n}\n\n","author_short":["Betti, A.","Reynaert, M.","Ossenkoppele, T.","Oortwijn, Y.","Salway, A.","Bloem, J."],"key":"betti_expert_2020","id":"betti_expert_2020","bibbaseid":"betti-reynaert-ossenkoppele-oortwijn-salway-bloem-expertconceptmodelinggroundtruthconstructionforwordembeddingsevaluationinconceptfocuseddomains-2020","role":"author","urls":{"Paper":"https://www.aclweb.org/anthology/2020.coling-main.586"},"metadata":{"authorlinks":{"betti, a":"https://bibbase.org/show?bib=https%3A%2F%2Fapi.zotero.org%2Fgroups%2F214700%2Fitems%3Fkey%3Dbi2Q7duoPuqjf6lgym4TgM83%26format%3Dbibtex%26limit%3D100&group0=year&css=www.axiom.humanities.uva.nl/axiom.css"}},"downloads":4},"bibtype":"inproceedings","biburl":"https://api.zotero.org/groups/2493581/items?key=esJ5YGDX6948PQKQSPOfhZpO&format=bibtex&limit=100","creationDate":"2021-01-28T08:29:41.521Z","downloads":4,"keywords":[],"search_terms":["expert","concept","modeling","ground","truth","construction","word","embeddings","evaluation","concept","focused","domains","betti","reynaert","ossenkoppele","oortwijn","salway","bloem"],"title":"Expert Concept-Modeling Ground Truth Construction for Word Embeddings Evaluation in Concept-Focused Domains","year":2020,"dataSources":["ZHdsvgePffKRdgqdo","ATwFjrHEsqA4GDy8B","W9xJhCsMa9uhxgDWu","9kNXu23ABH7j4wxXZ","zDZS7QvC6khJT2mcu"]}