Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.

Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. Kung, T. H, Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. PLOS digital health, 2(2):1, February, 2023.

Paper doi abstract bibtex

We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.

@article{kung_performance_2023,
	title = {Performance of {ChatGPT} on {USMLE}: {Potential} for {AI}-assisted medical education using large language models.},
	volume = {2},
	url = {https://www.proquest.com/scholarly-journals/performance-chatgpt-on-usmle-potential-ai/docview/2779350110/se-2?accountid=14542},
	doi = {10.1371/journal.pdig.0000198},
	abstract = {We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.},
	language = {English},
	number = {2},
	journal = {PLOS digital health},
	author = {Kung, Tiffany H and Cheatham, Morgan and Medenilla, Arielle and Sillos, Czarina and De Leon, Lorie and Elepaño, Camille and Madriaga, Maria and Aggabao, Rimel and Diaz-Candido, Giezel and Maningo, James and Tseng, Victor},
	month = feb,
	year = {2023},
	pages = {1},
	annote = {Fecha de creación - 2023-02-22},
	annote = {Fecha de revisión - 2023-02-27},
	annote = {SuppNotes - Conflict of Interest: The authors have declared that no competing interests exist. Cited By: Crit Care Med. 2018 Jun;46(6):e481-e488 [29419557] Mayo Clin Proc. 2013 Aug;88(8):790-8 [23871230] Sci Transl Med. 2021 Mar 24;13(586): [33762434] J Med Internet Res. 2020 Oct 22;22(10):e20346 [33090118] Nat Med. 2020 Jun;26(6):900-908 [32424212] JAMA Netw Open. 2022 Sep 1;5(9):e2233946 [36173632] Acad Med. 2021 Jan 1;96(1):113-117 [33394663] Nat Mater. 2019 May;18(5):410-414 [31000806] Acad Med. 2017 Nov;92(11S Association of American Medical Colleges Learn Serve Lead: P):S67-S74 [29065026] Rend Accad Naz XL. 1965-1966;16-17:189-219 [11632843] NPJ Digit Med. 2019 Jun 7;2:48 [31304394] Elife. 2019 Jun 11;8: [31182188] JAMA. 2016 Dec 13;316(22):2402-2410 [27898976] Nat Med. 2022 May;28(5):924-933 [35585198] PLoS One. 2019 Mar 6;14(3):e0213258 [30840682] JMIR Form Res. 2020 Jun 5;4(6):e16670 [32442148] Trans Am Clin Climatol Assoc. 2011;122:48-58 [21686208] EBioMedicine. 2019 Aug;46:27-29 [31303500]},
	annote = {Última actualización - 2023-02-27},
	file = {PubMed Central Full Text PDF:files/8583/Kung et al. - 2023 - Performance of ChatGPT on USMLE Potential for AI-.pdf:application/pdf},
}

Downloads: 0

{"_id":"tiZ2DN2wefXSmuk9C","bibbaseid":"kung-cheatham-medenilla-sillos-deleon-elepao-madriaga-aggabao-etal-performanceofchatgptonusmlepotentialforaiassistedmedicaleducationusinglargelanguagemodels-2023","author_short":["Kung, T. H","Cheatham, M.","Medenilla, A.","Sillos, C.","De Leon, L.","Elepaño, C.","Madriaga, M.","Aggabao, R.","Diaz-Candido, G.","Maningo, J.","Tseng, V."],"bibdata":{"bibtype":"article","type":"article","title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.","volume":"2","url":"https://www.proquest.com/scholarly-journals/performance-chatgpt-on-usmle-potential-ai/docview/2779350110/se-2?accountid=14542","doi":"10.1371/journal.pdig.0000198","abstract":"We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.","language":"English","number":"2","journal":"PLOS digital health","author":[{"propositions":[],"lastnames":["Kung"],"firstnames":["Tiffany","H"],"suffixes":[]},{"propositions":[],"lastnames":["Cheatham"],"firstnames":["Morgan"],"suffixes":[]},{"propositions":[],"lastnames":["Medenilla"],"firstnames":["Arielle"],"suffixes":[]},{"propositions":[],"lastnames":["Sillos"],"firstnames":["Czarina"],"suffixes":[]},{"propositions":[],"lastnames":["De","Leon"],"firstnames":["Lorie"],"suffixes":[]},{"propositions":[],"lastnames":["Elepaño"],"firstnames":["Camille"],"suffixes":[]},{"propositions":[],"lastnames":["Madriaga"],"firstnames":["Maria"],"suffixes":[]},{"propositions":[],"lastnames":["Aggabao"],"firstnames":["Rimel"],"suffixes":[]},{"propositions":[],"lastnames":["Diaz-Candido"],"firstnames":["Giezel"],"suffixes":[]},{"propositions":[],"lastnames":["Maningo"],"firstnames":["James"],"suffixes":[]},{"propositions":[],"lastnames":["Tseng"],"firstnames":["Victor"],"suffixes":[]}],"month":"February","year":"2023","pages":"1","annote":"Última actualización - 2023-02-27","file":"PubMed Central Full Text PDF:files/8583/Kung et al. - 2023 - Performance of ChatGPT on USMLE Potential for AI-.pdf:application/pdf","bibtex":"@article{kung_performance_2023,\n\ttitle = {Performance of {ChatGPT} on {USMLE}: {Potential} for {AI}-assisted medical education using large language models.},\n\tvolume = {2},\n\turl = {https://www.proquest.com/scholarly-journals/performance-chatgpt-on-usmle-potential-ai/docview/2779350110/se-2?accountid=14542},\n\tdoi = {10.1371/journal.pdig.0000198},\n\tabstract = {We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.},\n\tlanguage = {English},\n\tnumber = {2},\n\tjournal = {PLOS digital health},\n\tauthor = {Kung, Tiffany H and Cheatham, Morgan and Medenilla, Arielle and Sillos, Czarina and De Leon, Lorie and Elepaño, Camille and Madriaga, Maria and Aggabao, Rimel and Diaz-Candido, Giezel and Maningo, James and Tseng, Victor},\n\tmonth = feb,\n\tyear = {2023},\n\tpages = {1},\n\tannote = {Fecha de creación - 2023-02-22},\n\tannote = {Fecha de revisión - 2023-02-27},\n\tannote = {SuppNotes - Conflict of Interest: The authors have declared that no competing interests exist. Cited By: Crit Care Med. 2018 Jun;46(6):e481-e488 [29419557] Mayo Clin Proc. 2013 Aug;88(8):790-8 [23871230] Sci Transl Med. 2021 Mar 24;13(586): [33762434] J Med Internet Res. 2020 Oct 22;22(10):e20346 [33090118] Nat Med. 2020 Jun;26(6):900-908 [32424212] JAMA Netw Open. 2022 Sep 1;5(9):e2233946 [36173632] Acad Med. 2021 Jan 1;96(1):113-117 [33394663] Nat Mater. 2019 May;18(5):410-414 [31000806] Acad Med. 2017 Nov;92(11S Association of American Medical Colleges Learn Serve Lead: P):S67-S74 [29065026] Rend Accad Naz XL. 1965-1966;16-17:189-219 [11632843] NPJ Digit Med. 2019 Jun 7;2:48 [31304394] Elife. 2019 Jun 11;8: [31182188] JAMA. 2016 Dec 13;316(22):2402-2410 [27898976] Nat Med. 2022 May;28(5):924-933 [35585198] PLoS One. 2019 Mar 6;14(3):e0213258 [30840682] JMIR Form Res. 2020 Jun 5;4(6):e16670 [32442148] Trans Am Clin Climatol Assoc. 2011;122:48-58 [21686208] EBioMedicine. 2019 Aug;46:27-29 [31303500]},\n\tannote = {Última actualización - 2023-02-27},\n\tfile = {PubMed Central Full Text PDF:files/8583/Kung et al. - 2023 - Performance of ChatGPT on USMLE Potential for AI-.pdf:application/pdf},\n}\n\n","author_short":["Kung, T. H","Cheatham, M.","Medenilla, A.","Sillos, C.","De Leon, L.","Elepaño, C.","Madriaga, M.","Aggabao, R.","Diaz-Candido, G.","Maningo, J.","Tseng, V."],"key":"kung_performance_2023","id":"kung_performance_2023","bibbaseid":"kung-cheatham-medenilla-sillos-deleon-elepao-madriaga-aggabao-etal-performanceofchatgptonusmlepotentialforaiassistedmedicaleducationusinglargelanguagemodels-2023","role":"author","urls":{"Paper":"https://www.proquest.com/scholarly-journals/performance-chatgpt-on-usmle-potential-ai/docview/2779350110/se-2?accountid=14542"},"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"https://bibbase.org/network/files/22WYpzbBvi3hDHX7Y","dataSources":["cYu6uhMkeFHgRrEty","hLMh7bwHyFsPNWAEL","LKW3iRvnztCpLNTW7","TLD9JxqHfSQQ4r268","X9BvByJrC3kGJexn8","iovNvcnNYDGJcuMq2","NjZJ5ZmWhTtMZBfje"],"keywords":[],"search_terms":["performance","chatgpt","usmle","potential","assisted","medical","education","using","large","language","models","kung","cheatham","medenilla","sillos","de leon","elepaño","madriaga","aggabao","diaz-candido","maningo","tseng"],"title":"Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.","year":2023}