SEFLAG: Systematic Evaluation Framework for NLP Models and Datasets in Latin and Ancient Greek. Schulz, K. & Deichsler, F. November, 2024. ![SEFLAG: Systematic Evaluation Framework for NLP Models and Datasets in Latin and Ancient Greek [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex 1 download The poster presents SEFLAG, a Systematic Evaluation Framework for NLP models and datasets in Latin and Ancient Greek, developed at Humboldt-Universität zu Berlin. It addresses three core research questions: helping literary scholars select suitable NLP models, systematically documenting language resources, and unifying similar but distinct annotation schemas. SEFLAG integrates components such as model evaluation, data curation, and documentation, utilizing tools like spaCy, flair, Hugging Face, and Zenodo. It supports tasks including named entity recognition, lemmatization, and dependency parsing. A key feature is mapping between different annotation schemas to ensure comparability across resources. Evaluation metrics (such as F1, accuracy) show performance results for both Latin and Ancient Greek across several models and datasets. Challenges in this domain include linguistic variation, limited resources, interoperability issues, and the need for sustainable, interdisciplinary research. SEFLAG contributes solutions such as publishing model cards and datasheets, using Linked Data for evaluation results, and offering case-specific mappings. Future plans include expanding to more tasks, models, and datasets, creating educational materials on NLP evaluation, and fully integrating the framework into the Daidalos research infrastructure. All resources are open-access, with code and evaluation data available online.
@misc{schulz_seflag_2024,
title = {{SEFLAG}: {Systematic} {Evaluation} {Framework} for {NLP} {Models} and {Datasets} in {Latin} and {Ancient} {Greek}},
shorttitle = {{SEFLAG}},
url = {https://zenodo.org/records/15790925},
abstract = {The poster presents SEFLAG, a Systematic Evaluation Framework for NLP models and datasets in Latin and Ancient Greek, developed at Humboldt-Universität zu Berlin. It addresses three core research questions: helping literary scholars select suitable NLP models, systematically documenting language resources, and unifying similar but distinct annotation schemas.
SEFLAG integrates components such as model evaluation, data curation, and documentation, utilizing tools like spaCy, flair, Hugging Face, and Zenodo. It supports tasks including named entity recognition, lemmatization, and dependency parsing. A key feature is mapping between different annotation schemas to ensure comparability across resources. Evaluation metrics (such as F1, accuracy) show performance results for both Latin and Ancient Greek across several models and datasets.
Challenges in this domain include linguistic variation, limited resources, interoperability issues, and the need for sustainable, interdisciplinary research. SEFLAG contributes solutions such as publishing model cards and datasheets, using Linked Data for evaluation results, and offering case-specific mappings.
Future plans include expanding to more tasks, models, and datasets, creating educational materials on NLP evaluation, and fully integrating the framework into the Daidalos research infrastructure. All resources are open-access, with code and evaluation data available online.},
language = {eng},
urldate = {2025-07-02},
author = {Schulz, Konstantin and Deichsler, Florian},
month = nov,
year = {2024},
doi = {10.5281/zenodo.15790925},
keywords = {Artificial Intelligence, Artificial intelligence, Languages and literature, Literature, Literature studies, Literature study, Natural Language Processing, Natural language processing},
}
Downloads: 1
{"_id":"5wp5gbqn2f8rbFfeq","bibbaseid":"schulz-deichsler-seflagsystematicevaluationframeworkfornlpmodelsanddatasetsinlatinandancientgreek-2024","author_short":["Schulz, K.","Deichsler, F."],"bibdata":{"bibtype":"misc","type":"misc","title":"SEFLAG: Systematic Evaluation Framework for NLP Models and Datasets in Latin and Ancient Greek","shorttitle":"SEFLAG","url":"https://zenodo.org/records/15790925","abstract":"The poster presents SEFLAG, a Systematic Evaluation Framework for NLP models and datasets in Latin and Ancient Greek, developed at Humboldt-Universität zu Berlin. It addresses three core research questions: helping literary scholars select suitable NLP models, systematically documenting language resources, and unifying similar but distinct annotation schemas. SEFLAG integrates components such as model evaluation, data curation, and documentation, utilizing tools like spaCy, flair, Hugging Face, and Zenodo. It supports tasks including named entity recognition, lemmatization, and dependency parsing. A key feature is mapping between different annotation schemas to ensure comparability across resources. Evaluation metrics (such as F1, accuracy) show performance results for both Latin and Ancient Greek across several models and datasets. Challenges in this domain include linguistic variation, limited resources, interoperability issues, and the need for sustainable, interdisciplinary research. SEFLAG contributes solutions such as publishing model cards and datasheets, using Linked Data for evaluation results, and offering case-specific mappings. Future plans include expanding to more tasks, models, and datasets, creating educational materials on NLP evaluation, and fully integrating the framework into the Daidalos research infrastructure. All resources are open-access, with code and evaluation data available online.","language":"eng","urldate":"2025-07-02","author":[{"propositions":[],"lastnames":["Schulz"],"firstnames":["Konstantin"],"suffixes":[]},{"propositions":[],"lastnames":["Deichsler"],"firstnames":["Florian"],"suffixes":[]}],"month":"November","year":"2024","doi":"10.5281/zenodo.15790925","keywords":"Artificial Intelligence, Artificial intelligence, Languages and literature, Literature, Literature studies, Literature study, Natural Language Processing, Natural language processing","bibtex":"@misc{schulz_seflag_2024,\n\ttitle = {{SEFLAG}: {Systematic} {Evaluation} {Framework} for {NLP} {Models} and {Datasets} in {Latin} and {Ancient} {Greek}},\n\tshorttitle = {{SEFLAG}},\n\turl = {https://zenodo.org/records/15790925},\n\tabstract = {The poster presents SEFLAG, a Systematic Evaluation Framework for NLP models and datasets in Latin and Ancient Greek, developed at Humboldt-Universität zu Berlin. It addresses three core research questions: helping literary scholars select suitable NLP models, systematically documenting language resources, and unifying similar but distinct annotation schemas.\n\nSEFLAG integrates components such as model evaluation, data curation, and documentation, utilizing tools like spaCy, flair, Hugging Face, and Zenodo. It supports tasks including named entity recognition, lemmatization, and dependency parsing. A key feature is mapping between different annotation schemas to ensure comparability across resources. Evaluation metrics (such as F1, accuracy) show performance results for both Latin and Ancient Greek across several models and datasets.\n\nChallenges in this domain include linguistic variation, limited resources, interoperability issues, and the need for sustainable, interdisciplinary research. SEFLAG contributes solutions such as publishing model cards and datasheets, using Linked Data for evaluation results, and offering case-specific mappings.\n\nFuture plans include expanding to more tasks, models, and datasets, creating educational materials on NLP evaluation, and fully integrating the framework into the Daidalos research infrastructure. All resources are open-access, with code and evaluation data available online.},\n\tlanguage = {eng},\n\turldate = {2025-07-02},\n\tauthor = {Schulz, Konstantin and Deichsler, Florian},\n\tmonth = nov,\n\tyear = {2024},\n\tdoi = {10.5281/zenodo.15790925},\n\tkeywords = {Artificial Intelligence, Artificial intelligence, Languages and literature, Literature, Literature studies, Literature study, Natural Language Processing, Natural language processing},\n}\n\n","author_short":["Schulz, K.","Deichsler, F."],"key":"schulz_seflag_2024","id":"schulz_seflag_2024","bibbaseid":"schulz-deichsler-seflagsystematicevaluationframeworkfornlpmodelsanddatasetsinlatinandancientgreek-2024","role":"author","urls":{"Paper":"https://zenodo.org/records/15790925"},"keyword":["Artificial Intelligence","Artificial intelligence","Languages and literature","Literature","Literature studies","Literature study","Natural Language Processing","Natural language processing"],"metadata":{"authorlinks":{}},"downloads":1},"bibtype":"misc","biburl":"https://api.zotero.org/users/912485/collections/VL2FMYT9/items?key=tJHJRX0dfdnwe3iuxbtyH4ht&format=bibtex&limit=100","dataSources":["nsAxma8Gv9c6fai5n","JFDnASMkoQCjjGL8E"],"keywords":["artificial intelligence","artificial intelligence","languages and literature","literature","literature studies","literature study","natural language processing","natural language processing"],"search_terms":["seflag","systematic","evaluation","framework","nlp","models","datasets","latin","ancient","greek","schulz","deichsler"],"title":"SEFLAG: Systematic Evaluation Framework for NLP Models and Datasets in Latin and Ancient Greek","year":2024,"downloads":1}