LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation. Lee, D., Khanna, R., Lin, B. Y., Lee, S., Ye, Q., Boschee, E., Neves, L., & Ren, X. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 372–379, Online, July, 2020. Association for Computational Linguistics. ![LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from, and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks – thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.
@inproceedings{lee-etal-2020-lean,
title = "{LEAN}-{LIFE}: A Label-Efficient Annotation Framework Towards Learning from Explanation",
author = "Lee, Dong-Ho and
Khanna, Rahul and
Lin, Bill Yuchen and
Lee, Seyeon and
Ye, Qinyuan and
Boschee, Elizabeth and
Neves, Leonardo and
Ren, Xiang",
booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations",
month = jul,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.acl-demos.42",
doi = "10.18653/v1/2020.acl-demos.42",
pages = "372--379",
abstract = "Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from, and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks {--} thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.",
}
Downloads: 0
{"_id":"QTb5Qba7T4ZvRhS32","bibbaseid":"lee-khanna-lin-lee-ye-boschee-neves-ren-leanlifealabelefficientannotationframeworktowardslearningfromexplanation-2020","author_short":["Lee, D.","Khanna, R.","Lin, B. Y.","Lee, S.","Ye, Q.","Boschee, E.","Neves, L.","Ren, X."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation","author":[{"propositions":[],"lastnames":["Lee"],"firstnames":["Dong-Ho"],"suffixes":[]},{"propositions":[],"lastnames":["Khanna"],"firstnames":["Rahul"],"suffixes":[]},{"propositions":[],"lastnames":["Lin"],"firstnames":["Bill","Yuchen"],"suffixes":[]},{"propositions":[],"lastnames":["Lee"],"firstnames":["Seyeon"],"suffixes":[]},{"propositions":[],"lastnames":["Ye"],"firstnames":["Qinyuan"],"suffixes":[]},{"propositions":[],"lastnames":["Boschee"],"firstnames":["Elizabeth"],"suffixes":[]},{"propositions":[],"lastnames":["Neves"],"firstnames":["Leonardo"],"suffixes":[]},{"propositions":[],"lastnames":["Ren"],"firstnames":["Xiang"],"suffixes":[]}],"booktitle":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations","month":"July","year":"2020","address":"Online","publisher":"Association for Computational Linguistics","url":"https://www.aclweb.org/anthology/2020.acl-demos.42","doi":"10.18653/v1/2020.acl-demos.42","pages":"372–379","abstract":"Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from, and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks – thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.","bibtex":"@inproceedings{lee-etal-2020-lean,\r\n title = \"{LEAN}-{LIFE}: A Label-Efficient Annotation Framework Towards Learning from Explanation\",\r\n author = \"Lee, Dong-Ho and\r\n Khanna, Rahul and\r\n Lin, Bill Yuchen and\r\n Lee, Seyeon and\r\n Ye, Qinyuan and\r\n Boschee, Elizabeth and\r\n Neves, Leonardo and\r\n Ren, Xiang\",\r\n booktitle = \"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations\",\r\n month = jul,\r\n year = \"2020\",\r\n address = \"Online\",\r\n publisher = \"Association for Computational Linguistics\",\r\n url = \"https://www.aclweb.org/anthology/2020.acl-demos.42\",\r\n doi = \"10.18653/v1/2020.acl-demos.42\",\r\n pages = \"372--379\",\r\n abstract = \"Successfully training a deep neural network demands a huge corpus of labeled data. However, each label only provides limited information to learn from, and collecting the requisite number of labels involves massive human effort. In this work, we introduce LEAN-LIFE, a web-based, Label-Efficient AnnotatioN framework for sequence labeling and classification tasks, with an easy-to-use UI that not only allows an annotator to provide the needed labels for a task but also enables LearnIng From Explanations for each labeling decision. Such explanations enable us to generate useful additional labeled data from unlabeled instances, bolstering the pool of available training data. On three popular NLP tasks (named entity recognition, relation extraction, sentiment analysis), we find that using this enhanced supervision allows our models to surpass competitive baseline F1 scores by more than 5-10 percentage points, while using 2X times fewer labeled instances. Our framework is the first to utilize this enhanced supervision technique and does so for three important tasks {--} thus providing improved annotation recommendations to users and an ability to build datasets of (data, label, explanation) triples instead of the regular (data, label) pair.\",\r\n}\r\n\r\n\r\n\r\n\r\n\r\n","author_short":["Lee, D.","Khanna, R.","Lin, B. Y.","Lee, S.","Ye, Q.","Boschee, E.","Neves, L.","Ren, X."],"bibbaseid":"lee-khanna-lin-lee-ye-boschee-neves-ren-leanlifealabelefficientannotationframeworktowardslearningfromexplanation-2020","role":"author","urls":{"Paper":"https://www.aclweb.org/anthology/2020.acl-demos.42"},"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://bibbase.org/f/SKBwv9n9W4YYh9SfC/boschee-2023.bib","dataSources":["eShyn9ox8xhiJBHq2","6xESkCofuRDYuE4dM","dfnxo2P7wcDdnT5Pz"],"keywords":[],"search_terms":["lean","life","label","efficient","annotation","framework","towards","learning","explanation","lee","khanna","lin","lee","ye","boschee","neves","ren"],"title":"LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation","year":2020}