The Mismeasurement of Science. Lawrence, P. A. 17(15):R583-R585. ![The Mismeasurement of Science [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex [Excerpt:Impact factors and citations] Crucially, impact factors are distorted by positive feedback – many citations are not based on reading the paper but by reading other papers, particularly reviews. One study even suggested that, of cited articles, only some 20\,% had actually been read. [...] Nevertheless, citations are now being used to make quantitative comparisons between scientists. [...] [Changes in behaviour] Unfortunately, the use of these measures is having damaging effects on perceptions and on behaviour; these I list below. Please note that I am not saying that all science has gone rotten, I am describing trends and extremes; there are many principled researchers and teachers, but they are having an increasingly difficult time. [::] First, there is the nature of scientific reporting. It has become vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial (because of the weight put on such papers by grant-awarding bodies). Two or three such papers can make the difference between unemployment and tenure. [...] [::] Second, trying to meet the measures involves changing research strategy: risks should not be taken as this can mean long periods trying out new things, good for the originality of research but bad if a grant has to be renewed. [...] [::] Third, there is the presentation of the results: hype your work, slice the findings up as much as possible (four papers good, two papers bad), compress the results (most top journals have little space, a typical Nature letter now has the density of a black hole), simplify your conclusions but complexify the material (more difficult for reviewers to fault it!), mine rich sources of data, even if they lack originality. Most damagingly, I find it has become profitable to ignore or hide results that do not fit with the story being sold – a mix of evidence tends to make a paper look messy and lower its appeal. [::] Fourth, there is the way that science is done and papers are authored. These measures are pushing people into having larger groups. It is a simple matter of arithmetic. Since the group leader authors all the papers, the more people, the more papers. If a larger proportion of young scientists in a larger group fail, as I suspect, this is not recorded. And because no account is taken of wasted lives and broken dreams, these failures do not make a group leader look less productive. [...] [::] Fifth, leaving your lab to network is being rewarded; it can pay to build tacit webs of mutual support amongst colleagues, some of whom will review your papers. [...] [::] Sixth, there is an unexpected effect. The struggle to survive in modern science, the open and public nature of that competition, and the advantages bestowed on those who are prepared to show off and to exploit others have acted against modest and gentle people of all kinds – yet there is no evidence, presumption or likelihood that less pushy people are less creative. As less aggressive people are predominantly women [...] it should be no surprise that, in spite of an increased proportion of women entering biomedical research as students, there has been little, if any, increase in the representation of women at the top. Gentle people of both sexes vote with their feet and leave a profession that they, correctly, perceive to discriminate against them. Not only do we lose many original researchers, I think science would flourish more in an understanding and empathetic workplace. [:Who is to blame and what to do?] The main villains are fashion, the management cult and the politics of our time, all of which favour numerical evaluation of 'performance' and reward compliance. Over recent years, within governments and outside them, people have lost sight of the primary purposes of institutions, and a growing obsession with internal processes has driven more and more bureaucracy – such as increasingly complex grant applications and baroque research assessment exercises – at the expense of research effort. This bureaucracy is placing heavy demands on scientists that lay waste their sense of purpose and attack their self-esteem. But scientists of all ranks, senior as well as junior, are also to blame as we have meekly allowed this to happen. But can we now start to fight back? We need to raise awareness of the problems and make changes locally. [...] [] [...]
@article{lawrenceMismeasurementScience2007,
title = {The Mismeasurement of Science},
author = {Lawrence, Peter A.},
date = {2007-08},
journaltitle = {Current Biology},
volume = {17},
pages = {R583-R585},
issn = {0960-9822},
doi = {10.1016/j.cub.2007.06.014},
url = {http://mfkp.org/INRMM/article/1572875},
abstract = {[Excerpt:Impact factors and citations] Crucially, impact factors are distorted by positive feedback -- many citations are not based on reading the paper but by reading other papers, particularly reviews. One study even suggested that, of cited articles, only some 20\,\% had actually been read. [...] Nevertheless, citations are now being used to make quantitative comparisons between scientists. [...]
[Changes in behaviour] Unfortunately, the use of these measures is having damaging effects on perceptions and on behaviour; these I list below. Please note that I am not saying that all science has gone rotten, I am describing trends and extremes; there are many principled researchers and teachers, but they are having an increasingly difficult time.
[::] First, there is the nature of scientific reporting. It has become vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial (because of the weight put on such papers by grant-awarding bodies). Two or three such papers can make the difference between unemployment and tenure. [...]
[::] Second, trying to meet the measures involves changing research strategy: risks should not be taken as this can mean long periods trying out new things, good for the originality of research but bad if a grant has to be renewed. [...]
[::] Third, there is the presentation of the results: hype your work, slice the findings up as much as possible (four papers good, two papers bad), compress the results (most top journals have little space, a typical Nature letter now has the density of a black hole), simplify your conclusions but complexify the material (more difficult for reviewers to fault it!), mine rich sources of data, even if they lack originality. Most damagingly, I find it has become profitable to ignore or hide results that do not fit with the story being sold -- a mix of evidence tends to make a paper look messy and lower its appeal.
[::] Fourth, there is the way that science is done and papers are authored. These measures are pushing people into having larger groups. It is a simple matter of arithmetic. Since the group leader authors all the papers, the more people, the more papers. If a larger proportion of young scientists in a larger group fail, as I suspect, this is not recorded. And because no account is taken of wasted lives and broken dreams, these failures do not make a group leader look less productive. [...]
[::] Fifth, leaving your lab to network is being rewarded; it can pay to build tacit webs of mutual support amongst colleagues, some of whom will review your papers. [...]
[::] Sixth, there is an unexpected effect. The struggle to survive in modern science, the open and public nature of that competition, and the advantages bestowed on those who are prepared to show off and to exploit others have acted against modest and gentle people of all kinds -- yet there is no evidence, presumption or likelihood that less pushy people are less creative. As less aggressive people are predominantly women [...] it should be no surprise that, in spite of an increased proportion of women entering biomedical research as students, there has been little, if any, increase in the representation of women at the top. Gentle people of both sexes vote with their feet and leave a profession that they, correctly, perceive to discriminate against them. Not only do we lose many original researchers, I think science would flourish more in an understanding and empathetic workplace.
[:Who is to blame and what to do?]
The main villains are fashion, the management cult and the politics of our time, all of which favour numerical evaluation of 'performance' and reward compliance. Over recent years, within governments and outside them, people have lost sight of the primary purposes of institutions, and a growing obsession with internal processes has driven more and more bureaucracy -- such as increasingly complex grant applications and baroque research assessment exercises -- at the expense of research effort. This bureaucracy is placing heavy demands on scientists that lay waste their sense of purpose and attack their self-esteem. But scientists of all ranks, senior as well as junior, are also to blame as we have meekly allowed this to happen. But can we now start to fight back? We need to raise awareness of the problems and make changes locally. [...]
[] [...]},
keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-1572875,~to-add-doi-URL,education,ethics,gender-biases,impact-factor,peer-review,publication-bias,publish-or-perish,research-funding,research-metrics,science-ethics,scientific-communication,scientific-creativity,scientific-misconduct},
number = {15}
}
Downloads: 0
{"_id":"xTtAyQCE8nqRuxjTu","bibbaseid":"lawrence-themismeasurementofscience","authorIDs":[],"author_short":["Lawrence, P. A."],"bibdata":{"bibtype":"article","type":"article","title":"The Mismeasurement of Science","author":[{"propositions":[],"lastnames":["Lawrence"],"firstnames":["Peter","A."],"suffixes":[]}],"date":"2007-08","journaltitle":"Current Biology","volume":"17","pages":"R583-R585","issn":"0960-9822","doi":"10.1016/j.cub.2007.06.014","url":"http://mfkp.org/INRMM/article/1572875","abstract":"[Excerpt:Impact factors and citations] Crucially, impact factors are distorted by positive feedback – many citations are not based on reading the paper but by reading other papers, particularly reviews. One study even suggested that, of cited articles, only some 20\\,% had actually been read. [...] Nevertheless, citations are now being used to make quantitative comparisons between scientists. [...] [Changes in behaviour] Unfortunately, the use of these measures is having damaging effects on perceptions and on behaviour; these I list below. Please note that I am not saying that all science has gone rotten, I am describing trends and extremes; there are many principled researchers and teachers, but they are having an increasingly difficult time. [::] First, there is the nature of scientific reporting. It has become vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial (because of the weight put on such papers by grant-awarding bodies). Two or three such papers can make the difference between unemployment and tenure. [...] [::] Second, trying to meet the measures involves changing research strategy: risks should not be taken as this can mean long periods trying out new things, good for the originality of research but bad if a grant has to be renewed. [...] [::] Third, there is the presentation of the results: hype your work, slice the findings up as much as possible (four papers good, two papers bad), compress the results (most top journals have little space, a typical Nature letter now has the density of a black hole), simplify your conclusions but complexify the material (more difficult for reviewers to fault it!), mine rich sources of data, even if they lack originality. Most damagingly, I find it has become profitable to ignore or hide results that do not fit with the story being sold – a mix of evidence tends to make a paper look messy and lower its appeal. [::] Fourth, there is the way that science is done and papers are authored. These measures are pushing people into having larger groups. It is a simple matter of arithmetic. Since the group leader authors all the papers, the more people, the more papers. If a larger proportion of young scientists in a larger group fail, as I suspect, this is not recorded. And because no account is taken of wasted lives and broken dreams, these failures do not make a group leader look less productive. [...] [::] Fifth, leaving your lab to network is being rewarded; it can pay to build tacit webs of mutual support amongst colleagues, some of whom will review your papers. [...] [::] Sixth, there is an unexpected effect. The struggle to survive in modern science, the open and public nature of that competition, and the advantages bestowed on those who are prepared to show off and to exploit others have acted against modest and gentle people of all kinds – yet there is no evidence, presumption or likelihood that less pushy people are less creative. As less aggressive people are predominantly women [...] it should be no surprise that, in spite of an increased proportion of women entering biomedical research as students, there has been little, if any, increase in the representation of women at the top. Gentle people of both sexes vote with their feet and leave a profession that they, correctly, perceive to discriminate against them. Not only do we lose many original researchers, I think science would flourish more in an understanding and empathetic workplace. [:Who is to blame and what to do?] The main villains are fashion, the management cult and the politics of our time, all of which favour numerical evaluation of 'performance' and reward compliance. Over recent years, within governments and outside them, people have lost sight of the primary purposes of institutions, and a growing obsession with internal processes has driven more and more bureaucracy – such as increasingly complex grant applications and baroque research assessment exercises – at the expense of research effort. This bureaucracy is placing heavy demands on scientists that lay waste their sense of purpose and attack their self-esteem. But scientists of all ranks, senior as well as junior, are also to blame as we have meekly allowed this to happen. But can we now start to fight back? We need to raise awareness of the problems and make changes locally. [...] [] [...]","keywords":"*imported-from-citeulike-INRMM,~INRMM-MiD:c-1572875,~to-add-doi-URL,education,ethics,gender-biases,impact-factor,peer-review,publication-bias,publish-or-perish,research-funding,research-metrics,science-ethics,scientific-communication,scientific-creativity,scientific-misconduct","number":"15","bibtex":"@article{lawrenceMismeasurementScience2007,\n title = {The Mismeasurement of Science},\n author = {Lawrence, Peter A.},\n date = {2007-08},\n journaltitle = {Current Biology},\n volume = {17},\n pages = {R583-R585},\n issn = {0960-9822},\n doi = {10.1016/j.cub.2007.06.014},\n url = {http://mfkp.org/INRMM/article/1572875},\n abstract = {[Excerpt:Impact factors and citations] Crucially, impact factors are distorted by positive feedback -- many citations are not based on reading the paper but by reading other papers, particularly reviews. One study even suggested that, of cited articles, only some 20\\,\\% had actually been read. [...] Nevertheless, citations are now being used to make quantitative comparisons between scientists. [...]\n\n[Changes in behaviour] Unfortunately, the use of these measures is having damaging effects on perceptions and on behaviour; these I list below. Please note that I am not saying that all science has gone rotten, I am describing trends and extremes; there are many principled researchers and teachers, but they are having an increasingly difficult time.\n\n[::] First, there is the nature of scientific reporting. It has become vital to get papers into high impact-factor journals; just one such paper can change the prospects of a postdoc from nonexistent to substantial (because of the weight put on such papers by grant-awarding bodies). Two or three such papers can make the difference between unemployment and tenure. [...]\n\n[::] Second, trying to meet the measures involves changing research strategy: risks should not be taken as this can mean long periods trying out new things, good for the originality of research but bad if a grant has to be renewed. [...]\n\n[::] Third, there is the presentation of the results: hype your work, slice the findings up as much as possible (four papers good, two papers bad), compress the results (most top journals have little space, a typical Nature letter now has the density of a black hole), simplify your conclusions but complexify the material (more difficult for reviewers to fault it!), mine rich sources of data, even if they lack originality. Most damagingly, I find it has become profitable to ignore or hide results that do not fit with the story being sold -- a mix of evidence tends to make a paper look messy and lower its appeal.\n\n[::] Fourth, there is the way that science is done and papers are authored. These measures are pushing people into having larger groups. It is a simple matter of arithmetic. Since the group leader authors all the papers, the more people, the more papers. If a larger proportion of young scientists in a larger group fail, as I suspect, this is not recorded. And because no account is taken of wasted lives and broken dreams, these failures do not make a group leader look less productive. [...]\n\n[::] Fifth, leaving your lab to network is being rewarded; it can pay to build tacit webs of mutual support amongst colleagues, some of whom will review your papers. [...]\n\n[::] Sixth, there is an unexpected effect. The struggle to survive in modern science, the open and public nature of that competition, and the advantages bestowed on those who are prepared to show off and to exploit others have acted against modest and gentle people of all kinds -- yet there is no evidence, presumption or likelihood that less pushy people are less creative. As less aggressive people are predominantly women [...] it should be no surprise that, in spite of an increased proportion of women entering biomedical research as students, there has been little, if any, increase in the representation of women at the top. Gentle people of both sexes vote with their feet and leave a profession that they, correctly, perceive to discriminate against them. Not only do we lose many original researchers, I think science would flourish more in an understanding and empathetic workplace.\n\n[:Who is to blame and what to do?]\n\nThe main villains are fashion, the management cult and the politics of our time, all of which favour numerical evaluation of 'performance' and reward compliance. Over recent years, within governments and outside them, people have lost sight of the primary purposes of institutions, and a growing obsession with internal processes has driven more and more bureaucracy -- such as increasingly complex grant applications and baroque research assessment exercises -- at the expense of research effort. This bureaucracy is placing heavy demands on scientists that lay waste their sense of purpose and attack their self-esteem. But scientists of all ranks, senior as well as junior, are also to blame as we have meekly allowed this to happen. But can we now start to fight back? We need to raise awareness of the problems and make changes locally. [...]\n\n[] [...]},\n keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-1572875,~to-add-doi-URL,education,ethics,gender-biases,impact-factor,peer-review,publication-bias,publish-or-perish,research-funding,research-metrics,science-ethics,scientific-communication,scientific-creativity,scientific-misconduct},\n number = {15}\n}\n\n","author_short":["Lawrence, P. A."],"key":"lawrenceMismeasurementScience2007","id":"lawrenceMismeasurementScience2007","bibbaseid":"lawrence-themismeasurementofscience","role":"author","urls":{"Paper":"http://mfkp.org/INRMM/article/1572875"},"keyword":["*imported-from-citeulike-INRMM","~INRMM-MiD:c-1572875","~to-add-doi-URL","education","ethics","gender-biases","impact-factor","peer-review","publication-bias","publish-or-perish","research-funding","research-metrics","science-ethics","scientific-communication","scientific-creativity","scientific-misconduct"],"downloads":0},"bibtype":"article","biburl":"https://tmpfiles.org/dl/58794/INRMM.bib","creationDate":"2020-07-02T22:41:12.123Z","downloads":0,"keywords":["*imported-from-citeulike-inrmm","~inrmm-mid:c-1572875","~to-add-doi-url","education","ethics","gender-biases","impact-factor","peer-review","publication-bias","publish-or-perish","research-funding","research-metrics","science-ethics","scientific-communication","scientific-creativity","scientific-misconduct"],"search_terms":["mismeasurement","science","lawrence"],"title":"The Mismeasurement of Science","year":null,"dataSources":["DXuKbcZTirdigFKPF"]}