Transparency in Ecology and Evolution: Real Problems, Real Solutions. Parker, T. H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J. D., Chee, Y. E., Kelly, C. D., Gurevitch, J., & Nakagawa, S. 31(9):711–719.
Transparency in Ecology and Evolution: Real Problems, Real Solutions [link]Paper  doi  abstract   bibtex   
To make progress scientists need to know what other researchers have found and how they found it. However, transparency is often insufficient across much of ecology and evolution. Researchers often fail to report results and methods in detail sufficient to permit interpretation and meta-analysis, and many results go entirely unreported. Further, these unreported results are often a biased subset. Thus the conclusions we can draw from the published literature are themselves often biased and sometimes might be entirely incorrect. Fortunately there is a movement across empirical disciplines, and now within ecology and evolution, to shape editorial policies to better promote transparency. This can be done by either requiring more disclosure by scientists or by developing incentives to encourage disclosure. [::Trends] Evidence suggests that insufficient transparency is a problem across much of ecology and evolution. Results and methods are often reported in insufficient detail or go entirely unreported. Further, these unreported results are often a biased subset, thus substantially hampering interpretation and meta-analysis. [\n] Journals and other institutions, such as funding agencies, influence researchers' decisions about disseminating results. There is a movement across empirical disciplines, including ecology and evolution, to shape institutional policies to better promote transparency. [\n] Institutions can promote transparency by requiring or encouraging more disclosure, as with the now-familiar data archiving, or by developing an incentive structure promoting disclosure, such as preregistration of studies and analysis plans. [Excerpt: Some solutions to improve transparency] [...] In November 2015, representatives (mostly editors-in-chief) from nearly 30 journals in ecology and evolution joined funding-agency panelists and other researchers to identify ways to improve transparency in these disciplines. At this workshop, strong support emerged for the recently introduced Transparency and Openness Promotion (TOP) framework (https://cos.io/top/). [...] [\n] [...] The TOP guideline titled 'analysis and design transparency' calls for discipline-specific guidance regarding what information should be disclosed in publications, and, to that end, the workshop produced a document 'Tools for Transparency in Ecology and Evolution' (TTEE; https://osf.io/g65cb/) that provides checklist questions that journals can provide to authors, reviewers, and editors to facilitate transparent reporting. Promoting more-thorough and consistent reporting of results and methods through TOP and TTEE should dramatically improve transparency, but here we also highlight two other TOP components that could have transformative impacts on our field. [::Preregistration], in which researchers register their study and data analysis plan before collecting data, can greatly improve transparency. Although requiring preregistration (as in clinical trial research) might thwart publication of valuable exploratory and serendipitous findings, encouraging preregistration where appropriate has large potential benefits. Most obviously, it makes unpublished results more discoverable, thus helping to reduce publication bias. Potentially more important, however, preregistration of analysis plans ensures that we can identify genuine a priori planned tests, helping to improve confidence in results because they are unlikely to derive from hidden multiple hypothesis testing and selective reporting. As preregistration becomes more common, results that do not come from preregistered analysis plans become viewed as exploratory, and thus provisional and less convincing than preregistered results, providing a strong incentive to preregister studies. We acknowledge that exploratory work is hugely important in ecology and evolutionary biology and we do not wish to impede it, but it should be more consistently identifiable and it should be followed-up with planned, ideally preregistered, tests. A common concern is that preregistration ignores the inevitable tweaking of methods that occurs as field projects evolve. However, alterations to methods or analysis plans can be justified in the published study [...]. Reviewers and editors can decide if the reported methods and analyses adhered closely enough to the preregistration to earn a preregistration badge (https://osf.io/tvyxz/wiki/home/). Further, preregistered analyses and exploratory results can be published in the same paper when the distinction between them is made clear. [...] [::The final TOP guideline promotes replications of previously published studies]. Replication to assess validity and generality of prior results is a core practice of science. Exact replication is not possible, especially in field studies, but various forms of replication, especially when combined with meta-analysis, are powerful tools for establishing the applicability of hypotheses [37. Unfortunately, institutional incentive structures often work strongly against replication in ecology and evolution, especially replications that seek to closely match methods as part of the process of assessing validity [37. Journals and funding bodies explicitly favor novelty. Of course progress requires novelty, but progress also requires rigorous evaluation of prior findings. Not all studies are of high priority for replication. The more interesting or important a finding, however, the more important it is to replicate that study. [\n] [...]
@article{parkerTransparencyEcologyEvolution2016,
  title = {Transparency in Ecology and Evolution: Real Problems, Real Solutions},
  author = {Parker, Timothy H. and Forstmeier, Wolfgang and Koricheva, Julia and Fidler, Fiona and Hadfield, Jarrod D. and Chee, Yung E. and Kelly, Clint D. and Gurevitch, Jessica and Nakagawa, Shinichi},
  date = {2016-09},
  journaltitle = {Trends in Ecology \& Evolution},
  volume = {31},
  pages = {711--719},
  issn = {0169-5347},
  doi = {10.1016/j.tree.2016.07.002},
  url = {http://mfkp.org/INRMM/article/14127512},
  abstract = {To make progress scientists need to know what other researchers have found and how they found it. However, transparency is often insufficient across much of ecology and evolution. Researchers often fail to report results and methods in detail sufficient to permit interpretation and meta-analysis, and many results go entirely unreported. Further, these unreported results are often a biased subset. Thus the conclusions we can draw from the published literature are themselves often biased and sometimes might be entirely incorrect. Fortunately there is a movement across empirical disciplines, and now within ecology and evolution, to shape editorial policies to better promote transparency. This can be done by either requiring more disclosure by scientists or by developing incentives to encourage disclosure.

[::Trends]

Evidence suggests that insufficient transparency is a problem across much of ecology and evolution. Results and methods are often reported in insufficient detail or go entirely unreported. Further, these unreported results are often a biased subset, thus substantially hampering interpretation and meta-analysis.

[\textbackslash n] Journals and other institutions, such as funding agencies, influence researchers' decisions about disseminating results. There is a movement across empirical disciplines, including ecology and evolution, to shape institutional policies to better promote transparency.

[\textbackslash n] Institutions can promote transparency by requiring or encouraging more disclosure, as with the now-familiar data archiving, or by developing an incentive structure promoting disclosure, such as preregistration of studies and analysis plans.

[Excerpt: Some solutions to improve transparency] 

[...] In November 2015, representatives (mostly editors-in-chief) from nearly 30 journals in ecology and evolution joined funding-agency panelists and other researchers to identify ways to improve transparency in these disciplines. At this workshop, strong support emerged for the recently introduced Transparency and Openness Promotion (TOP) framework (https://cos.io/top/). [...]

[\textbackslash n] [...] The TOP guideline titled 'analysis and design transparency' calls for discipline-specific guidance regarding what information should be disclosed in publications, and, to that end, the workshop produced a document 'Tools for Transparency in Ecology and Evolution' (TTEE; https://osf.io/g65cb/) that provides checklist questions that journals can provide to authors, reviewers, and editors to facilitate transparent reporting. Promoting more-thorough and consistent reporting of results and methods through TOP and TTEE should dramatically improve transparency, but here we also highlight two other TOP components that could have transformative impacts on our field.

[::Preregistration], in which researchers register their study and data analysis plan before collecting data, can greatly improve transparency. Although requiring preregistration (as in clinical trial research) might thwart publication of valuable exploratory and serendipitous findings, encouraging preregistration where appropriate has large potential benefits. Most obviously, it makes unpublished results more discoverable, thus helping to reduce publication bias. Potentially more important, however, preregistration of analysis plans ensures that we can identify genuine a priori planned tests, helping to improve confidence in results because they are unlikely to derive from hidden multiple hypothesis testing and selective reporting. As preregistration becomes more common, results that do not come from preregistered analysis plans become viewed as exploratory, and thus provisional and less convincing than preregistered results, providing a strong incentive to preregister studies. We acknowledge that exploratory work is hugely important in ecology and evolutionary biology and we do not wish to impede it, but it should be more consistently identifiable and it should be followed-up with planned, ideally preregistered, tests. A common concern is that preregistration ignores the inevitable tweaking of methods that occurs as field projects evolve. However, alterations to methods or analysis plans can be justified in the published study [...]. Reviewers and editors can decide if the reported methods and analyses adhered closely enough to the preregistration to earn a preregistration badge (https://osf.io/tvyxz/wiki/home/). Further, preregistered analyses and exploratory results can be published in the same paper when the distinction between them is made clear. [...]

[::The final TOP guideline promotes replications of previously published studies]. Replication to assess validity and generality of prior results is a core practice of science. Exact replication is not possible, especially in field studies, but various forms of replication, especially when combined with meta-analysis, are powerful tools for establishing the applicability of hypotheses [37. Unfortunately, institutional incentive structures often work strongly against replication in ecology and evolution, especially replications that seek to closely match methods as part of the process of assessing validity [37. Journals and funding bodies explicitly favor novelty. Of course progress requires novelty, but progress also requires rigorous evaluation of prior findings. Not all studies are of high priority for replication. The more interesting or important a finding, however, the more important it is to replicate that study. 

[\textbackslash n] [...]},
  keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-14127512,~to-add-doi-URL,ecology,epistemology,open-data,open-science,reproducibility,reproducible-research,research-management,rewarding-best-research-practices,science-ethics,transparency},
  number = {9}
}
Downloads: 0
About
Documentation
Keywords
Search
Users
Terms and Conditions of Use
Privacy Policy
Sitemap
© BibBase.org 2021