ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models. Verbakel, J. Y., Steyerberg, E. W., Uno, H., Cock, B. D., Wynants, L., Collins, G. S., & Calster, B. V. Journal of Clinical Epidemiology, 126:207–216, October, 2020. Publisher: Elsevier
ROC curves for clinical prediction models part 1. ROC plots showed no added value above the AUC when evaluating the performance of clinical prediction models [link]Paper  doi  abstract   bibtex   
\textlessh2\textgreaterAbstract\textless/h2\textgreater\textlessh3\textgreaterObjectives\textless/h3\textgreater\textlessp\textgreaterReceiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations.\textless/p\textgreater\textlessh3\textgreaterStudy Design and Setting\textless/h3\textgreater\textlessp\textgreaterWe conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data.\textless/p\textgreater\textlessh3\textgreaterResults\textless/h3\textgreater\textlessp\textgreaterOf 86 identified prediction modeling studies, 52 (60%) presented ROC curves without thresholds and one (1%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds.\textless/p\textgreater\textlessh3\textgreaterConclusion\textless/h3\textgreater\textlessp\textgreaterROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive.\textless/p\textgreater
@article{verbakel_roc_2020,
	title = {{ROC} curves for clinical prediction models part 1. {ROC} plots showed no added value above the {AUC} when evaluating the performance of clinical prediction models},
	volume = {126},
	issn = {0895-4356, 1878-5921},
	url = {https://www.jclinepi.com/article/S0895-4356(18)31004-7/abstract},
	doi = {10.1016/j.jclinepi.2020.01.028},
	abstract = {{\textless}h2{\textgreater}Abstract{\textless}/h2{\textgreater}{\textless}h3{\textgreater}Objectives{\textless}/h3{\textgreater}{\textless}p{\textgreater}Receiver operating characteristic (ROC) curves show how well a risk prediction model discriminates between patients with and without a condition. We aim to investigate how ROC curves are presented in the literature and discuss and illustrate their potential limitations.{\textless}/p{\textgreater}{\textless}h3{\textgreater}Study Design and Setting{\textless}/h3{\textgreater}{\textless}p{\textgreater}We conducted a pragmatic literature review of contemporary publications that externally validated clinical prediction models. We illustrated limitations of ROC curves using a testicular cancer case study and simulated data.{\textless}/p{\textgreater}{\textless}h3{\textgreater}Results{\textless}/h3{\textgreater}{\textless}p{\textgreater}Of 86 identified prediction modeling studies, 52 (60\%) presented ROC curves without thresholds and one (1\%) presented an ROC curve with only a few thresholds. We illustrate that ROC curves in their standard form withhold threshold information have an unstable shape even for the same area under the curve (AUC) and are problematic for comparing model performance conditional on threshold. We compare ROC curves with classification plots, which show sensitivity and specificity conditional on risk thresholds.{\textless}/p{\textgreater}{\textless}h3{\textgreater}Conclusion{\textless}/h3{\textgreater}{\textless}p{\textgreater}ROC curves do not offer more information than the AUC to indicate discriminative ability. To assess the model's performance for decision-making, results should be provided conditional on risk thresholds. Therefore, if discriminatory ability must be visualized, classification plots are attractive.{\textless}/p{\textgreater}},
	language = {English},
	urldate = {2020-12-08},
	journal = {Journal of Clinical Epidemiology},
	author = {Verbakel, Jan Y. and Steyerberg, Ewout W. and Uno, Hajime and Cock, Bavo De and Wynants, Laure and Collins, Gary S. and Calster, Ben Van},
	month = oct,
	year = {2020},
	pmid = {32712176},
	note = {Publisher: Elsevier},
	keywords = {roc},
	pages = {207--216},
}

Downloads: 0