It's Impossible to Conduct Research without Software, Say 7 out of 10 UK Researchers. Hettrick, S. Software and research, 5:1536+, 2014.
abstract   bibtex   
No one knows how much software is used in research. Look around any lab and you'll see software - both standard and bespoke - being used by all disciplines and seniorities of researchers. Software is clearly fundamental to research, but we can't prove this without evidence. And this lack of evidence is the reason why we ran a survey of researchers at 15 Russell Group universities to find out about their software use and background. [Excerpt: Headline figures] [::] 92\,% of academics use research software [::] 69\,% say that their research would not be practical without it [::] 56\,% develop their own software (worryingly, 21\,% of those have no training in software development [::] 70\,% of male researchers develop their own software, and only 30\,% of female researchers do so [Data] The data collected during this survey is available for download from Zenodo ("S.J. Hettrick et al, UK Research Software Survey 2014"\mbox\mbox, DOI:10.5281/zenodo.14809 ). [...] [] [...] [Software is far more important to research than anyone knows] If we do not know how much we rely on software, we cannot ensure that researchers have the tools and skills they need to stay at the forefront of research. We collected evidence - for the first time at this scale - of research software use, development, and training. [...] [] [...] [How many researchers use software?] It's not overstating the case to say that software is vital to research. If we were to magically remove software from research, 7 out of 10 researchers would be out of a job. [] 92\,% of respondents said they used research software. More importantly, 70\,% of respondents said that '' It would not be practical to conduct my work without software''. [] [...] [What software are people using?] A lot of different software is being used: we recorded 566 different packages - some of them have only one user within our surveyed community, some with many. The most popular packages are Matlab (20\,% of respondents use it), R (16\,%), SPSS (15\,%), then Excel (12\,%). [...] [A lot of researchers are developing their own software - even though they lack training] It's not just proprietary software, many researchers are developing their own code: 56\,% of them. This is great news, because the real power of software lies in developing it to allow you to do more in less time and make new research possible. [] Many people in the research community are developing their own software, is the development in safe hands? [] 55\,% of respondents have received some training in software development (15\,% self taught and 40\,% had received some form of taught course). Worryingly, 21\,% of respondents who develop their own software had no training in software development. That's one in five researchers developing software blind. [] Software that is developed without adequate training is unlikely to be reliable. Researchers are, by their very nature, intelligent people who learn new skills quickly, but there are many subtle pitfalls in developing good code (that is, code that won't later lead to paper retractions). And that's only the case for reliability! We want defensible results, which requires a whole swathe of skills related to producing reproducible code, and we want to protect the research investment, which requires yet more skills for writing reusable software. [Software development costs are not being included in bids] Many researchers believe that including costs for developing software in a proposal will weaken it. We've had steer from the Research Councils that this is not the case - something we're trying to persuade the research community to believe. But we may have our work cut out. [] When we asked the people who are responsible for writing proposals whether they had included costs for software development, 22\,% said that they had, 57\,% said they had not, and 20\,% said that they had not even though they knew software development would make up part of the bid! (Note that rounding errors make these figures sum to 99\,%.) [] [...] [What can you tell from a researcher's operating system of choice?] There is a difference, albeit not a great one, when it comes to simply using research software: 88\,% of Windows users are also users of research software, as compared to 93\,% for OS X and a remarkable 98\,% for Linux. [] When it comes to developing research software, the differences become apparent. Only 41\,% of Windows users develop research software, which again is slightly behind OS X at 53\,%. Linux users are in a field of their own: 90\,% of them develop their own research software. [...] [] [...]
@article{hettrickItImpossibleConduct2014,
  title = {It's Impossible to Conduct Research without Software, Say 7 out of 10 {{UK}} Researchers},
  author = {Hettrick, Simon},
  year = {2014},
  volume = {5},
  pages = {1536+},
  abstract = {No one knows how much software is used in research. Look around any lab and you'll see software - both standard and bespoke - being used by all disciplines and seniorities of researchers. Software is clearly fundamental to research, but we can't prove this without evidence. And this lack of evidence is the reason why we ran a survey of researchers at 15 Russell Group universities to find out about their software use and background.

[Excerpt: Headline figures]

[::] 92\,\% of academics use research software [::] 69\,\% say that their research would not be practical without it [::] 56\,\% develop their own software (worryingly, 21\,\% of those have no training in software development [::] 70\,\% of male researchers develop their own software, and only 30\,\% of female researchers do so

[Data]

The data collected during this survey is available for download from Zenodo ("S.J. Hettrick et al, UK Research Software Survey 2014"\mbox\mbox, DOI:10.5281/zenodo.14809 ). [...]

[] [...]

[Software is far more important to research than anyone knows]

If we do not know how much we rely on software, we cannot ensure that researchers have the tools and skills they need to stay at the forefront of research. We collected evidence - for the first time at this scale - of research software use, development, and training. [...]

[] [...]

[How many researchers use software?]

It's not overstating the case to say that software is vital to research. If we were to magically remove software from research, 7 out of 10 researchers would be out of a job.

[] 92\,\% of respondents said they used research software. More importantly, 70\,\% of respondents said that '' It would not be practical to conduct my work without software''.

[] [...]

[What software are people using?]

A lot of different software is being used: we recorded 566 different packages - some of them have only one user within our surveyed community, some with many. The most popular packages are Matlab (20\,\% of respondents use it), R (16\,\%), SPSS (15\,\%), then Excel (12\,\%). [...]

[A lot of researchers are developing their own software - even though they lack training]

It's not just proprietary software, many researchers are developing their own code: 56\,\% of them. This is great news, because the real power of software lies in developing it to allow you to do more in less time and make new research possible.

[] Many people in the research community are developing their own software, is the development in safe hands?

[] 55\,\% of respondents have received some training in software development (15\,\% self taught and 40\,\% had received some form of taught course). Worryingly, 21\,\% of respondents who develop their own software had no training in software development. That's one in five researchers developing software blind.

[] Software that is developed without adequate training is unlikely to be reliable. Researchers are, by their very nature, intelligent people who learn new skills quickly, but there are many subtle pitfalls in developing good code (that is, code that won't later lead to paper retractions). And that's only the case for reliability! We want defensible results, which requires a whole swathe of skills related to producing reproducible code, and we want to protect the research investment, which requires yet more skills for writing reusable software.

[Software development costs are not being included in bids]

Many researchers believe that including costs for developing software in a proposal will weaken it. We've had steer from the Research Councils that this is not the case - something we're trying to persuade the research community to believe. But we may have our work cut out.

[] When we asked the people who are responsible for writing proposals whether they had included costs for software development, 22\,\% said that they had, 57\,\% said they had not, and 20\,\% said that they had not even though they knew software development would make up part of the bid! (Note that rounding errors make these figures sum to 99\,\%.)

[] [...]

[What can you tell from a researcher's operating system of choice?]

There is a difference, albeit not a great one, when it comes to simply using research software: 88\,\% of Windows users are also users of research software, as compared to 93\,\% for OS X and a remarkable 98\,\% for Linux.

[] When it comes to developing research software, the differences become apparent. Only 41\,\% of Windows users develop research software, which again is slightly behind OS X at 53\,\%. Linux users are in a field of their own: 90\,\% of them develop their own research software. [...]

[] [...]},
  journal = {Software and research},
  keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-14219104,bias-disembodied-science-vs-computational-scholarship,computational-science,programming,research-management,research-metrics,survey,united-kingdom},
  lccn = {INRMM-MiD:c-14219104}
}