Conversational Automated Program Repair. Xia, C. S. & Zhang, L. arXiv.org, January, 2023. Place: Ithaca Publisher: Cornell University Library, arXiv.org![Conversational Automated Program Repair [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper abstract bibtex Automated Program Repair (APR) can help developers automatically generate patches for bugs. Due to the impressive performance obtained using Large Pre-Trained Language Models (LLMs) on many code related tasks, researchers have started to directly use LLMs for APR. However, prior approaches simply repeatedly sample the LLM given the same constructed input/prompt created from the original buggy code, which not only leads to generating the same incorrect patches repeatedly but also miss the critical information in testcases. To address these limitations, we propose conversational APR, a new paradigm for program repair that alternates between patch generation and validation in a conversational manner. In conversational APR, we iteratively build the input to the model by combining previously generated patches with validation feedback. As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test. We evaluate 10 different LLM including the newly developed ChatGPT model to demonstrate the improvement of conversational APR over the prior LLM for APR approach.
@article{xia_conversational_2023,
title = {Conversational {Automated} {Program} {Repair}},
url = {https://www.proquest.com/working-papers/conversational-automated-program-repair/docview/2771491018/se-2},
abstract = {Automated Program Repair (APR) can help developers automatically generate patches for bugs. Due to the impressive performance obtained using Large Pre-Trained Language Models (LLMs) on many code related tasks, researchers have started to directly use LLMs for APR. However, prior approaches simply repeatedly sample the LLM given the same constructed input/prompt created from the original buggy code, which not only leads to generating the same incorrect patches repeatedly but also miss the critical information in testcases. To address these limitations, we propose conversational APR, a new paradigm for program repair that alternates between patch generation and validation in a conversational manner. In conversational APR, we iteratively build the input to the model by combining previously generated patches with validation feedback. As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test. We evaluate 10 different LLM including the newly developed ChatGPT model to demonstrate the improvement of conversational APR over the prior LLM for APR approach.},
language = {English},
journal = {arXiv.org},
author = {Xia, Chunqiu Steven and Zhang, Lingming},
month = jan,
year = {2023},
note = {Place: Ithaca
Publisher: Cornell University Library, arXiv.org},
keywords = {Automation, Machine Learning, Business And Economics--Banking And Finance, Software Engineering, Feedback, Patches (structures)},
}
Downloads: 0
{"_id":"fASJqsCu7CQz5Ggf8","bibbaseid":"xia-zhang-conversationalautomatedprogramrepair-2023","author_short":["Xia, C. S.","Zhang, L."],"bibdata":{"bibtype":"article","type":"article","title":"Conversational Automated Program Repair","url":"https://www.proquest.com/working-papers/conversational-automated-program-repair/docview/2771491018/se-2","abstract":"Automated Program Repair (APR) can help developers automatically generate patches for bugs. Due to the impressive performance obtained using Large Pre-Trained Language Models (LLMs) on many code related tasks, researchers have started to directly use LLMs for APR. However, prior approaches simply repeatedly sample the LLM given the same constructed input/prompt created from the original buggy code, which not only leads to generating the same incorrect patches repeatedly but also miss the critical information in testcases. To address these limitations, we propose conversational APR, a new paradigm for program repair that alternates between patch generation and validation in a conversational manner. In conversational APR, we iteratively build the input to the model by combining previously generated patches with validation feedback. As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test. We evaluate 10 different LLM including the newly developed ChatGPT model to demonstrate the improvement of conversational APR over the prior LLM for APR approach.","language":"English","journal":"arXiv.org","author":[{"propositions":[],"lastnames":["Xia"],"firstnames":["Chunqiu","Steven"],"suffixes":[]},{"propositions":[],"lastnames":["Zhang"],"firstnames":["Lingming"],"suffixes":[]}],"month":"January","year":"2023","note":"Place: Ithaca Publisher: Cornell University Library, arXiv.org","keywords":"Automation, Machine Learning, Business And Economics–Banking And Finance, Software Engineering, Feedback, Patches (structures)","bibtex":"@article{xia_conversational_2023,\n\ttitle = {Conversational {Automated} {Program} {Repair}},\n\turl = {https://www.proquest.com/working-papers/conversational-automated-program-repair/docview/2771491018/se-2},\n\tabstract = {Automated Program Repair (APR) can help developers automatically generate patches for bugs. Due to the impressive performance obtained using Large Pre-Trained Language Models (LLMs) on many code related tasks, researchers have started to directly use LLMs for APR. However, prior approaches simply repeatedly sample the LLM given the same constructed input/prompt created from the original buggy code, which not only leads to generating the same incorrect patches repeatedly but also miss the critical information in testcases. To address these limitations, we propose conversational APR, a new paradigm for program repair that alternates between patch generation and validation in a conversational manner. In conversational APR, we iteratively build the input to the model by combining previously generated patches with validation feedback. As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test. We evaluate 10 different LLM including the newly developed ChatGPT model to demonstrate the improvement of conversational APR over the prior LLM for APR approach.},\n\tlanguage = {English},\n\tjournal = {arXiv.org},\n\tauthor = {Xia, Chunqiu Steven and Zhang, Lingming},\n\tmonth = jan,\n\tyear = {2023},\n\tnote = {Place: Ithaca\nPublisher: Cornell University Library, arXiv.org},\n\tkeywords = {Automation, Machine Learning, Business And Economics--Banking And Finance, Software Engineering, Feedback, Patches (structures)},\n}\n\n","author_short":["Xia, C. S.","Zhang, L."],"key":"xia_conversational_2023","id":"xia_conversational_2023","bibbaseid":"xia-zhang-conversationalautomatedprogramrepair-2023","role":"author","urls":{"Paper":"https://www.proquest.com/working-papers/conversational-automated-program-repair/docview/2771491018/se-2"},"keyword":["Automation","Machine Learning","Business And Economics–Banking And Finance","Software Engineering","Feedback","Patches (structures)"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"https://bibbase.org/network/files/srCiZydJWRzXD39Ko","dataSources":["cYu6uhMkeFHgRrEty","hLMh7bwHyFsPNWAEL","LKW3iRvnztCpLNTW7","TLD9JxqHfSQQ4r268","X9BvByJrC3kGJexn8","iovNvcnNYDGJcuMq2","NjZJ5ZmWhTtMZBfje"],"keywords":["automation","machine learning","business and economics–banking and finance","software engineering","feedback","patches (structures)"],"search_terms":["conversational","automated","program","repair","xia","zhang"],"title":"Conversational Automated Program Repair","year":2023}