Top Tips to Make Your Research Irreproducible. Chue Hong, N. P., Crick, T., Gent, I. P., Kotthoff, L., & Takeda, K.
Top Tips to Make Your Research Irreproducible [link]Paper  abstract   bibtex   
It is an unfortunate convention of science that research should pretend to be reproducible; our top tips will help you mitigate this fussy conventionality, enabling you to enthusiastically showcase your irreproducible work. [Excerpt] [...] Irreproducibility is the default setting for all of science, and irreproducible research is particularly common across the computational sciences. [...] By following our starter tips, you can ensure that if your work is wrong, nobody will be able to check it; if it is correct, you will make everyone else do disproportionately more work than you to build upon it. [::1 Think 'Big Picture'] People are interested in the science, not the dull experimental setup, so don't describe it. If necessary, camouflage this absence with brief, high-level details of insignificant aspects of your methodology. [::2. Be abstract] Pseudo-code is a great way of communicating ideas quickly and clearly while giving readers no chance to understand the subtle implementation details (particularly the custom toolchains and manual interventions) that actually make it work. [::3. Short and sweet] Any limitations of your methods or proofs will be obvious to the careful reader, so there is no need to waste space on making them explicit. However much work it takes colleagues to fill in the gaps, you will still get the credit if you just say you have amazing experiments or proofs (with a hat-tip to Pierre de Fermat: ” Cuius rei demonstrationem mirabilem sane detexi hanc marginis exiguitas non caperet.”). [::4. The deficit model] You're the expert in the domain, only you can define what algorithms and data to run experiments with. In the unhappy circumstance that your methods do not do well on community curated benchmarks, you should create your own bespoke benchmarks and use those (and preferably not make them available to others). [::5. Don't share] Doing so only makes it easier for other people to scoop your research ideas, understand how your code actually works3 instead of why you say it does, or worst of all to understand that your code doesn't actually work at all. [] [...] We make a simple conjecture: an experiment that is irreproducible is exactly equivalent to an experiment that was never carried out at all. [...] [] We close with a mantra for scientists interested in irreproducibility: After Publishing Research, Irreproducibility Lets False Observations Obtain Longevity!
@article{chuehongTopTipsMake2015,
  title = {Top Tips to Make Your Research Irreproducible},
  author = {Chue Hong, Neil P. and Crick, Tom and Gent, Ian P. and Kotthoff, Lars and Takeda, Kenji},
  date = {2015-04},
  url = {http://mfkp.org/INRMM/article/13581223},
  abstract = {It is an unfortunate convention of science that research should pretend to be reproducible; our top tips will help you mitigate this fussy conventionality, enabling you to enthusiastically showcase your irreproducible work.

[Excerpt] [...] Irreproducibility is the default setting for all of science, and irreproducible research is particularly common across the computational sciences. [...] By following our starter tips, you can ensure that if your work is wrong, nobody will be able to check it; if it is correct, you will make everyone else do disproportionately more work than you to build upon it.

[::1 Think 'Big Picture'] People are interested in the science, not the dull experimental setup, so don't describe it. If necessary, camouflage this absence with brief, high-level details of insignificant aspects of your methodology.

[::2. Be abstract] Pseudo-code is a great way of communicating ideas quickly and clearly while giving readers no chance to understand the subtle implementation details (particularly the custom toolchains and manual interventions) that actually make it work.

[::3. Short and sweet] Any limitations of your methods or proofs will be obvious to the careful reader, so there is no need to waste space on making them explicit. However much work it takes colleagues to fill in the gaps, you will still get the credit if you just say you have amazing experiments or proofs (with a hat-tip to Pierre de Fermat: ” Cuius rei demonstrationem mirabilem sane detexi hanc marginis exiguitas non caperet.”).

[::4. The deficit model] You're the expert in the domain, only you can define what algorithms and data to run experiments with. In the unhappy circumstance that your methods do not do well on community curated benchmarks, you should create your own bespoke benchmarks and use those (and preferably not make them available to others).

[::5. Don't share] Doing so only makes it easier for other people to scoop your research ideas, understand how your code actually works3 instead of why you say it does, or worst of all to understand that your code doesn't actually work at all.

[] [...] We make a simple conjecture: an experiment that is irreproducible is exactly equivalent to an experiment that was never carried out at all. [...]

[] We close with a mantra for scientists interested in irreproducibility: After Publishing Research, Irreproducibility Lets False Observations Obtain Longevity!},
  archivePrefix = {arXiv},
  eprint = {1504.00062},
  eprinttype = {arxiv},
  keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-13581223,automation-irony,check-list,false-observations-propagation,reproducibility,reproducible-research,uncertainty-propagation}
}
Downloads: 0
About
Documentation
Keywords
Search
Users
Terms and Conditions of Use
Privacy Policy
Sitemap
© BibBase.org 2021