Automated identification of eviction status from electronic health record notes

Automated identification of eviction status from electronic health record notes. Yao, Z., Tsai, J., Liu, W., Levy, D. A, Druhl, E., Reisman, J. I, & Yu, H. Journal of the American Medical Informatics Association, May, 2023.

Paper doi abstract bibtex

Evictions are important social and behavioral determinants of health. Evictions are associated with a cascade of negative events that can lead to unemployment, housing insecurity/homelessness, long-term poverty, and mental health problems. In this study, we developed a natural language processing system to automatically detect eviction status from electronic health record (EHR) notes.We first defined eviction status (eviction presence and eviction period) and then annotated eviction status in 5000 EHR notes from the Veterans Health Administration (VHA). We developed a novel model, KIRESH, that has shown to substantially outperform other state-of-the-art models such as fine-tuning pretrained language models like BioBERT and Bio_ClinicalBERT. Moreover, we designed a novel prompt to further improve the model performance by using the intrinsic connection between the 2 subtasks of eviction presence and period prediction. Finally, we used the Temperature Scaling-based Calibration on our KIRESH-Prompt method to avoid overconfidence issues arising from the imbalance dataset.KIRESH-Prompt substantially outperformed strong baseline models including fine-tuning the Bio_ClinicalBERT model to achieve 0.74672 MCC, 0.71153 Macro-F1, and 0.83396 Micro-F1 in predicting eviction period and 0.66827 MCC, 0.62734 Macro-F1, and 0.7863 Micro-F1 in predicting eviction presence. We also conducted additional experiments on a benchmark social determinants of health (SBDH) dataset to demonstrate the generalizability of our methods.KIRESH-Prompt has substantially improved eviction status classification. We plan to deploy KIRESH-Prompt to the VHA EHRs as an eviction surveillance system to help address the US Veterans’ housing insecurity.

@article{yao_automated_2023,
	title = {Automated identification of eviction status from electronic health record notes},
	issn = {1527-974X},
	url = {https://doi.org/10.1093/jamia/ocad081},
	doi = {10.1093/jamia/ocad081},
	abstract = {Evictions are important social and behavioral determinants of health. Evictions are associated with a cascade of negative events that can lead to unemployment, housing insecurity/homelessness, long-term poverty, and mental health problems. In this study, we developed a natural language processing system to automatically detect eviction status from electronic health record (EHR) notes.We first defined eviction status (eviction presence and eviction period) and then annotated eviction status in 5000 EHR notes from the Veterans Health Administration (VHA). We developed a novel model, KIRESH, that has shown to substantially outperform other state-of-the-art models such as fine-tuning pretrained language models like BioBERT and Bio\_ClinicalBERT. Moreover, we designed a novel prompt to further improve the model performance by using the intrinsic connection between the 2 subtasks of eviction presence and period prediction. Finally, we used the Temperature Scaling-based Calibration on our KIRESH-Prompt method to avoid overconfidence issues arising from the imbalance dataset.KIRESH-Prompt substantially outperformed strong baseline models including fine-tuning the Bio\_ClinicalBERT model to achieve 0.74672 MCC, 0.71153 Macro-F1, and 0.83396 Micro-F1 in predicting eviction period and 0.66827 MCC, 0.62734 Macro-F1, and 0.7863 Micro-F1 in predicting eviction presence. We also conducted additional experiments on a benchmark social determinants of health (SBDH) dataset to demonstrate the generalizability of our methods.KIRESH-Prompt has substantially improved eviction status classification. We plan to deploy KIRESH-Prompt to the VHA EHRs as an eviction surveillance system to help address the US Veterans’ housing insecurity.},
	urldate = {2023-05-19},
	journal = {Journal of the American Medical Informatics Association},
	author = {Yao, Zonghai and Tsai, Jack and Liu, Weisong and Levy, David A and Druhl, Emily and Reisman, Joel I and Yu, Hong},
	month = may,
	year = {2023},
	keywords = {Computer Science - Computation and Language},
	pages = {ocad081},
}

Downloads: 0

{"_id":"Hjk7Fd65SGtMveLMC","bibbaseid":"yao-tsai-liu-levy-druhl-reisman-yu-automatedidentificationofevictionstatusfromelectronichealthrecordnotes-2023","author_short":["Yao, Z.","Tsai, J.","Liu, W.","Levy, D. A","Druhl, E.","Reisman, J. I","Yu, H."],"bibdata":{"bibtype":"article","type":"article","title":"Automated identification of eviction status from electronic health record notes","issn":"1527-974X","url":"https://doi.org/10.1093/jamia/ocad081","doi":"10.1093/jamia/ocad081","abstract":"Evictions are important social and behavioral determinants of health. Evictions are associated with a cascade of negative events that can lead to unemployment, housing insecurity/homelessness, long-term poverty, and mental health problems. In this study, we developed a natural language processing system to automatically detect eviction status from electronic health record (EHR) notes.We first defined eviction status (eviction presence and eviction period) and then annotated eviction status in 5000 EHR notes from the Veterans Health Administration (VHA). We developed a novel model, KIRESH, that has shown to substantially outperform other state-of-the-art models such as fine-tuning pretrained language models like BioBERT and Bio_ClinicalBERT. Moreover, we designed a novel prompt to further improve the model performance by using the intrinsic connection between the 2 subtasks of eviction presence and period prediction. Finally, we used the Temperature Scaling-based Calibration on our KIRESH-Prompt method to avoid overconfidence issues arising from the imbalance dataset.KIRESH-Prompt substantially outperformed strong baseline models including fine-tuning the Bio_ClinicalBERT model to achieve 0.74672 MCC, 0.71153 Macro-F1, and 0.83396 Micro-F1 in predicting eviction period and 0.66827 MCC, 0.62734 Macro-F1, and 0.7863 Micro-F1 in predicting eviction presence. We also conducted additional experiments on a benchmark social determinants of health (SBDH) dataset to demonstrate the generalizability of our methods.KIRESH-Prompt has substantially improved eviction status classification. We plan to deploy KIRESH-Prompt to the VHA EHRs as an eviction surveillance system to help address the US Veterans’ housing insecurity.","urldate":"2023-05-19","journal":"Journal of the American Medical Informatics Association","author":[{"propositions":[],"lastnames":["Yao"],"firstnames":["Zonghai"],"suffixes":[]},{"propositions":[],"lastnames":["Tsai"],"firstnames":["Jack"],"suffixes":[]},{"propositions":[],"lastnames":["Liu"],"firstnames":["Weisong"],"suffixes":[]},{"propositions":[],"lastnames":["Levy"],"firstnames":["David","A"],"suffixes":[]},{"propositions":[],"lastnames":["Druhl"],"firstnames":["Emily"],"suffixes":[]},{"propositions":[],"lastnames":["Reisman"],"firstnames":["Joel","I"],"suffixes":[]},{"propositions":[],"lastnames":["Yu"],"firstnames":["Hong"],"suffixes":[]}],"month":"May","year":"2023","keywords":"Computer Science - Computation and Language","pages":"ocad081","bibtex":"@article{yao_automated_2023,\n\ttitle = {Automated identification of eviction status from electronic health record notes},\n\tissn = {1527-974X},\n\turl = {https://doi.org/10.1093/jamia/ocad081},\n\tdoi = {10.1093/jamia/ocad081},\n\tabstract = {Evictions are important social and behavioral determinants of health. Evictions are associated with a cascade of negative events that can lead to unemployment, housing insecurity/homelessness, long-term poverty, and mental health problems. In this study, we developed a natural language processing system to automatically detect eviction status from electronic health record (EHR) notes.We first defined eviction status (eviction presence and eviction period) and then annotated eviction status in 5000 EHR notes from the Veterans Health Administration (VHA). We developed a novel model, KIRESH, that has shown to substantially outperform other state-of-the-art models such as fine-tuning pretrained language models like BioBERT and Bio\\_ClinicalBERT. Moreover, we designed a novel prompt to further improve the model performance by using the intrinsic connection between the 2 subtasks of eviction presence and period prediction. Finally, we used the Temperature Scaling-based Calibration on our KIRESH-Prompt method to avoid overconfidence issues arising from the imbalance dataset.KIRESH-Prompt substantially outperformed strong baseline models including fine-tuning the Bio\\_ClinicalBERT model to achieve 0.74672 MCC, 0.71153 Macro-F1, and 0.83396 Micro-F1 in predicting eviction period and 0.66827 MCC, 0.62734 Macro-F1, and 0.7863 Micro-F1 in predicting eviction presence. We also conducted additional experiments on a benchmark social determinants of health (SBDH) dataset to demonstrate the generalizability of our methods.KIRESH-Prompt has substantially improved eviction status classification. We plan to deploy KIRESH-Prompt to the VHA EHRs as an eviction surveillance system to help address the US Veterans’ housing insecurity.},\n\turldate = {2023-05-19},\n\tjournal = {Journal of the American Medical Informatics Association},\n\tauthor = {Yao, Zonghai and Tsai, Jack and Liu, Weisong and Levy, David A and Druhl, Emily and Reisman, Joel I and Yu, Hong},\n\tmonth = may,\n\tyear = {2023},\n\tkeywords = {Computer Science - Computation and Language},\n\tpages = {ocad081},\n}\n\n","author_short":["Yao, Z.","Tsai, J.","Liu, W.","Levy, D. A","Druhl, E.","Reisman, J. I","Yu, H."],"key":"yao_automated_2023","id":"yao_automated_2023","bibbaseid":"yao-tsai-liu-levy-druhl-reisman-yu-automatedidentificationofevictionstatusfromelectronichealthrecordnotes-2023","role":"author","urls":{"Paper":"https://doi.org/10.1093/jamia/ocad081"},"keyword":["Computer Science - Computation and Language"],"metadata":{"authorlinks":{}},"html":""},"bibtype":"article","biburl":"http://fenway.cs.uml.edu/papers/pubs-all.bib","dataSources":["TqaA9miSB65nRfS5H"],"keywords":["computer science - computation and language"],"search_terms":["automated","identification","eviction","status","electronic","health","record","notes","yao","tsai","liu","levy","druhl","reisman","yu"],"title":"Automated identification of eviction status from electronic health record notes","year":2023}