A Tool for Generating Exceptional Behavior Tests With Large Language Models

A Tool for Generating Exceptional Behavior Tests With Large Language Models. Zhong, L., Yuan, S., Zhang, J., Liu, Y., Nie, P., Li, J. J., & Gligoric, M. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering, pages 1193–1197, Clarion Hotel Trondheim Trondheim Norway, June, 2025. ACM.

Paper doi abstract bibtex

Exceptional behavior tests (EBTs) are crucial in software development for verifying that code correctly handles unwanted events and throws appropriate exceptions. However, prior research has shown that developers often prioritize testing “happy paths”, i.e., paths without unwanted events, over exceptional scenarios. We present exLong, a tool that automatically generates EBTs to address this gap. exLong leverages a large language model (LLM) fine-tuned from CodeLlama and incorporates reasoning about exception-throwing traces, conditional expressions that guard throw statements, and non-exceptional behavior tests that execute similar traces. Our demonstration video illustrates how exLong can effectively assist developers in creating comprehensive EBTs for their project (available at https://youtu.be/Jro8kMgplZk).

@inproceedings{zhong_tool_2025,
	address = {Clarion Hotel Trondheim Trondheim Norway},
	title = {A {Tool} for {Generating} {Exceptional} {Behavior} {Tests} {With} {Large} {Language} {Models}},
	isbn = {979-8-4007-1276-0},
	url = {https://dl.acm.org/doi/10.1145/3696630.3728608},
	doi = {10.1145/3696630.3728608},
	abstract = {Exceptional behavior tests (EBTs) are crucial in software development for verifying that code correctly handles unwanted events and throws appropriate exceptions. However, prior research has shown that developers often prioritize testing “happy paths”, i.e., paths without unwanted events, over exceptional scenarios. We present exLong, a tool that automatically generates EBTs to address this gap. exLong leverages a large language model (LLM) fine-tuned from CodeLlama and incorporates reasoning about exception-throwing traces, conditional expressions that guard throw statements, and non-exceptional behavior tests that execute similar traces. Our demonstration video illustrates how exLong can effectively assist developers in creating comprehensive EBTs for their project (available at https://youtu.be/Jro8kMgplZk).},
	language = {en},
	urldate = {2025-08-28},
	booktitle = {Proceedings of the 33rd {ACM} {International} {Conference} on the {Foundations} of {Software} {Engineering}},
	publisher = {ACM},
	author = {Zhong, Linghan and Yuan, Samuel and Zhang, Jiyang and Liu, Yu and Nie, Pengyu and Li, Junyi Jessy and Gligoric, Milos},
	month = jun,
	year = {2025},
	pages = {1193--1197},
}

Downloads: 0

{"_id":"zGWAbvfKxJ39ynd7E","bibbaseid":"zhong-yuan-zhang-liu-nie-li-gligoric-atoolforgeneratingexceptionalbehaviortestswithlargelanguagemodels-2025","author_short":["Zhong, L.","Yuan, S.","Zhang, J.","Liu, Y.","Nie, P.","Li, J. J.","Gligoric, M."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"Clarion Hotel Trondheim Trondheim Norway","title":"A Tool for Generating Exceptional Behavior Tests With Large Language Models","isbn":"979-8-4007-1276-0","url":"https://dl.acm.org/doi/10.1145/3696630.3728608","doi":"10.1145/3696630.3728608","abstract":"Exceptional behavior tests (EBTs) are crucial in software development for verifying that code correctly handles unwanted events and throws appropriate exceptions. However, prior research has shown that developers often prioritize testing “happy paths”, i.e., paths without unwanted events, over exceptional scenarios. We present exLong, a tool that automatically generates EBTs to address this gap. exLong leverages a large language model (LLM) fine-tuned from CodeLlama and incorporates reasoning about exception-throwing traces, conditional expressions that guard throw statements, and non-exceptional behavior tests that execute similar traces. Our demonstration video illustrates how exLong can effectively assist developers in creating comprehensive EBTs for their project (available at https://youtu.be/Jro8kMgplZk).","language":"en","urldate":"2025-08-28","booktitle":"Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering","publisher":"ACM","author":[{"propositions":[],"lastnames":["Zhong"],"firstnames":["Linghan"],"suffixes":[]},{"propositions":[],"lastnames":["Yuan"],"firstnames":["Samuel"],"suffixes":[]},{"propositions":[],"lastnames":["Zhang"],"firstnames":["Jiyang"],"suffixes":[]},{"propositions":[],"lastnames":["Liu"],"firstnames":["Yu"],"suffixes":[]},{"propositions":[],"lastnames":["Nie"],"firstnames":["Pengyu"],"suffixes":[]},{"propositions":[],"lastnames":["Li"],"firstnames":["Junyi","Jessy"],"suffixes":[]},{"propositions":[],"lastnames":["Gligoric"],"firstnames":["Milos"],"suffixes":[]}],"month":"June","year":"2025","pages":"1193–1197","bibtex":"@inproceedings{zhong_tool_2025,\n\taddress = {Clarion Hotel Trondheim Trondheim Norway},\n\ttitle = {A {Tool} for {Generating} {Exceptional} {Behavior} {Tests} {With} {Large} {Language} {Models}},\n\tisbn = {979-8-4007-1276-0},\n\turl = {https://dl.acm.org/doi/10.1145/3696630.3728608},\n\tdoi = {10.1145/3696630.3728608},\n\tabstract = {Exceptional behavior tests (EBTs) are crucial in software development for verifying that code correctly handles unwanted events and throws appropriate exceptions. However, prior research has shown that developers often prioritize testing “happy paths”, i.e., paths without unwanted events, over exceptional scenarios. We present exLong, a tool that automatically generates EBTs to address this gap. exLong leverages a large language model (LLM) fine-tuned from CodeLlama and incorporates reasoning about exception-throwing traces, conditional expressions that guard throw statements, and non-exceptional behavior tests that execute similar traces. Our demonstration video illustrates how exLong can effectively assist developers in creating comprehensive EBTs for their project (available at https://youtu.be/Jro8kMgplZk).},\n\tlanguage = {en},\n\turldate = {2025-08-28},\n\tbooktitle = {Proceedings of the 33rd {ACM} {International} {Conference} on the {Foundations} of {Software} {Engineering}},\n\tpublisher = {ACM},\n\tauthor = {Zhong, Linghan and Yuan, Samuel and Zhang, Jiyang and Liu, Yu and Nie, Pengyu and Li, Junyi Jessy and Gligoric, Milos},\n\tmonth = jun,\n\tyear = {2025},\n\tpages = {1193--1197},\n}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n","author_short":["Zhong, L.","Yuan, S.","Zhang, J.","Liu, Y.","Nie, P.","Li, J. J.","Gligoric, M."],"key":"zhong_tool_2025","id":"zhong_tool_2025","bibbaseid":"zhong-yuan-zhang-liu-nie-li-gligoric-atoolforgeneratingexceptionalbehaviortestswithlargelanguagemodels-2025","role":"author","urls":{"Paper":"https://dl.acm.org/doi/10.1145/3696630.3728608"},"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://bibbase.org/zotero-group/pratikmhatre/5933976","dataSources":["yJr5AAtJ5Sz3Q4WT4"],"keywords":[],"search_terms":["tool","generating","exceptional","behavior","tests","large","language","models","zhong","yuan","zhang","liu","nie","li","gligoric"],"title":"A Tool for Generating Exceptional Behavior Tests With Large Language Models","year":2025}