An exploratory test for an excess of significant findings. Ioannidis, J. P A. & Trikalinos, T. A. Clin Trials, 4(3):245–253, 2007. doi abstract bibtex The published clinical research literature may be distorted by the pursuit of statistically significant results.We aimed to develop a test to explore biases stemming from the pursuit of nominal statistical significance.The exploratory test evaluates whether there is a relative excess of formally significant findings in the published literature due to any reason (e.g., publication bias, selective analyses and outcome reporting, or fabricated data). The number of expected studies with statistically significant results is estimated and compared against the number of observed significant studies. The main application uses alpha = 0.05, but a range of alpha thresholds is also examined. Different values or prior distributions of the effect size are assumed. Given the typically low power (few studies per research question), the test may be best applied across domains of many meta-analyses that share common characteristics (interventions, outcomes, study populations, research environment).We evaluated illustratively eight meta-analyses of clinical trials with >50 studies each and 10 meta-analyses of clinical efficacy for neuroleptic agents in schizophrenia; the 10 meta-analyses were also examined as a composite domain. Different results were obtained against commonly used tests of publication bias. We demonstrated a clear or possible excess of significant studies in 6 of 8 large meta-analyses and in the wide domain of neuroleptic treatments.The proposed test is exploratory, may depend on prior assumptions, and should be applied cautiously.An excess of significant findings may be documented in some clinical research fields.
@Article{Ioannidis2007,
author = {Ioannidis, John P A. and Trikalinos, Thomas A.},
journal = {Clin Trials},
title = {An exploratory test for an excess of significant findings.},
year = {2007},
number = {3},
pages = {245--253},
volume = {4},
abstract = {The published clinical research literature may be distorted by the
pursuit of statistically significant results.We aimed to develop
a test to explore biases stemming from the pursuit of nominal statistical
significance.The exploratory test evaluates whether there is a relative
excess of formally significant findings in the published literature
due to any reason (e.g., publication bias, selective analyses and
outcome reporting, or fabricated data). The number of expected studies
with statistically significant results is estimated and compared
against the number of observed significant studies. The main application
uses alpha = 0.05, but a range of alpha thresholds is also examined.
Different values or prior distributions of the effect size are assumed.
Given the typically low power (few studies per research question),
the test may be best applied across domains of many meta-analyses
that share common characteristics (interventions, outcomes, study
populations, research environment).We evaluated illustratively eight
meta-analyses of clinical trials with >50 studies each and 10 meta-analyses
of clinical efficacy for neuroleptic agents in schizophrenia; the
10 meta-analyses were also examined as a composite domain. Different
results were obtained against commonly used tests of publication
bias. We demonstrated a clear or possible excess of significant studies
in 6 of 8 large meta-analyses and in the wide domain of neuroleptic
treatments.The proposed test is exploratory, may depend on prior
assumptions, and should be applied cautiously.An excess of significant
findings may be documented in some clinical research fields.},
doi = {10.1177/1740774507079441},
institution = {Clinical Trials and Evidence Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. jioannid@cc.uoi.gr},
keywords = {Antipsychotic Agents, therapeutic use; Bias (Epidemiology); Clinical Trials as Topic, methods/statistics /&/ numerical data; Data Interpretation, Statistical; Humans; Meta-Analysis as Topic; Probability; Schizophrenia, drug therapy},
language = {eng},
medline-pst = {ppublish},
pmid = {17715249},
timestamp = {2015.07.19},
}
Downloads: 0
{"_id":"aro6kyGS88JJwgBRL","bibbaseid":"ioannidis-trikalinos-anexploratorytestforanexcessofsignificantfindings-2007","author_short":["Ioannidis, J. P A.","Trikalinos, T. A."],"bibdata":{"bibtype":"article","type":"article","author":[{"propositions":[],"lastnames":["Ioannidis"],"firstnames":["John","P","A."],"suffixes":[]},{"propositions":[],"lastnames":["Trikalinos"],"firstnames":["Thomas","A."],"suffixes":[]}],"journal":"Clin Trials","title":"An exploratory test for an excess of significant findings.","year":"2007","number":"3","pages":"245–253","volume":"4","abstract":"The published clinical research literature may be distorted by the pursuit of statistically significant results.We aimed to develop a test to explore biases stemming from the pursuit of nominal statistical significance.The exploratory test evaluates whether there is a relative excess of formally significant findings in the published literature due to any reason (e.g., publication bias, selective analyses and outcome reporting, or fabricated data). The number of expected studies with statistically significant results is estimated and compared against the number of observed significant studies. The main application uses alpha = 0.05, but a range of alpha thresholds is also examined. Different values or prior distributions of the effect size are assumed. Given the typically low power (few studies per research question), the test may be best applied across domains of many meta-analyses that share common characteristics (interventions, outcomes, study populations, research environment).We evaluated illustratively eight meta-analyses of clinical trials with >50 studies each and 10 meta-analyses of clinical efficacy for neuroleptic agents in schizophrenia; the 10 meta-analyses were also examined as a composite domain. Different results were obtained against commonly used tests of publication bias. We demonstrated a clear or possible excess of significant studies in 6 of 8 large meta-analyses and in the wide domain of neuroleptic treatments.The proposed test is exploratory, may depend on prior assumptions, and should be applied cautiously.An excess of significant findings may be documented in some clinical research fields.","doi":"10.1177/1740774507079441","institution":"Clinical Trials and Evidence Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. jioannid@cc.uoi.gr","keywords":"Antipsychotic Agents, therapeutic use; Bias (Epidemiology); Clinical Trials as Topic, methods/statistics /&/ numerical data; Data Interpretation, Statistical; Humans; Meta-Analysis as Topic; Probability; Schizophrenia, drug therapy","language":"eng","medline-pst":"ppublish","pmid":"17715249","timestamp":"2015.07.19","bibtex":"@Article{Ioannidis2007,\n author = {Ioannidis, John P A. and Trikalinos, Thomas A.},\n journal = {Clin Trials},\n title = {An exploratory test for an excess of significant findings.},\n year = {2007},\n number = {3},\n pages = {245--253},\n volume = {4},\n abstract = {The published clinical research literature may be distorted by the\n\tpursuit of statistically significant results.We aimed to develop\n\ta test to explore biases stemming from the pursuit of nominal statistical\n\tsignificance.The exploratory test evaluates whether there is a relative\n\texcess of formally significant findings in the published literature\n\tdue to any reason (e.g., publication bias, selective analyses and\n\toutcome reporting, or fabricated data). The number of expected studies\n\twith statistically significant results is estimated and compared\n\tagainst the number of observed significant studies. The main application\n\tuses alpha = 0.05, but a range of alpha thresholds is also examined.\n\tDifferent values or prior distributions of the effect size are assumed.\n\tGiven the typically low power (few studies per research question),\n\tthe test may be best applied across domains of many meta-analyses\n\tthat share common characteristics (interventions, outcomes, study\n\tpopulations, research environment).We evaluated illustratively eight\n\tmeta-analyses of clinical trials with >50 studies each and 10 meta-analyses\n\tof clinical efficacy for neuroleptic agents in schizophrenia; the\n\t10 meta-analyses were also examined as a composite domain. Different\n\tresults were obtained against commonly used tests of publication\n\tbias. We demonstrated a clear or possible excess of significant studies\n\tin 6 of 8 large meta-analyses and in the wide domain of neuroleptic\n\ttreatments.The proposed test is exploratory, may depend on prior\n\tassumptions, and should be applied cautiously.An excess of significant\n\tfindings may be documented in some clinical research fields.},\n doi = {10.1177/1740774507079441},\n institution = {Clinical Trials and Evidence Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece. jioannid@cc.uoi.gr},\n keywords = {Antipsychotic Agents, therapeutic use; Bias (Epidemiology); Clinical Trials as Topic, methods/statistics /&/ numerical data; Data Interpretation, Statistical; Humans; Meta-Analysis as Topic; Probability; Schizophrenia, drug therapy},\n language = {eng},\n medline-pst = {ppublish},\n pmid = {17715249},\n timestamp = {2015.07.19},\n}\n\n","author_short":["Ioannidis, J. P A.","Trikalinos, T. A."],"key":"Ioannidis2007","id":"Ioannidis2007","bibbaseid":"ioannidis-trikalinos-anexploratorytestforanexcessofsignificantfindings-2007","role":"author","urls":{},"keyword":["Antipsychotic Agents","therapeutic use; Bias (Epidemiology); Clinical Trials as Topic","methods/statistics /&/ numerical data; Data Interpretation","Statistical; Humans; Meta-Analysis as Topic; Probability; Schizophrenia","drug therapy"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"https://endress.org/publications/ansgar.bib","dataSources":["xPGxHAeh3vZpx4yyE","TXa55dQbNoWnaGmMq"],"keywords":["antipsychotic agents","therapeutic use; bias (epidemiology); clinical trials as topic","methods/statistics /&/ numerical data; data interpretation","statistical; humans; meta-analysis as topic; probability; schizophrenia","drug therapy"],"search_terms":["exploratory","test","excess","significant","findings","ioannidis","trikalinos"],"title":"An exploratory test for an excess of significant findings.","year":2007}