Good Data Are Not Enough. Loeb, A. 539(7627):23–25. ![Good Data Are Not Enough [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex A vibrant scientific culture encourages many interpretations of evidence, argues Avi Loeb. [Excerpt] [...] Most research funding is allocated assuming that the highest-quality data will inevitably deliver useful scientific interpretation and theoretical concepts, which can be tested and refined by future data. [...] To truly move forward, free thought must be encouraged outside the mainstream. Multiple interpretations of existing data and alternative motivations for collecting new data must be supported. [...] [Blinkered view] Mayan cosmologists had high social status. They got generous support because of their promises to forecast the future. Cosmologists today collect vast amounts of exquisite data in surveys of large parts of the sky, costing billions of dollars. [] Such projects have a narrow aim – pinning down the parameters of one theoretical model. The model comprises an expanding Universe composed of dark matter, dark energy and normal matter (from which stars, planets and people are made), with initial conditions dictated by an early phase of rapid expansion called cosmic inflation. The data are reduced to a few numbers. Surprises in the rest are tossed away. [] I noticed this bias recently while assessing a PhD thesis. The student was asked to test whether a data set from a large cosmological survey was in line with the standard cosmological model. But when a discrepancy was found, the student's goal shifted to explaining why the data set was incomplete. In such a culture, the current model can never be ruled out, even though everyone knows that its major constituents [...] are not understood at a fundamental level. [] Instead, observers should present results in a theory-neutral way. [...] Vast data sets may contain evidence for unusual behaviour that was unanticipated when the projects were conceived. If all results are expected and planned for, babies may be thrown out with the bathwater. [...] [Alternative paths] The only way to work out whether we are on the wrong path is to encourage competing interpretations of the known data. [] I have been arguing for many years that funding agencies should promote the analysis of data for serendipitous purposes beyond major programmes and the mainstream dogma. [...] At a minimum, when funding is tight, a research frontier should maintain at least two ways of interpreting data so that new experiments will aim to select the correct one. [...] A healthy dialogue between different points of view should be fostered through multidisciplinary conferences that discuss conceptual issues, not just experimental results and phenomenology. A diversity of views fosters healthy progress and prevents stagnation. [...]
@article{loebGoodDataAre2016,
title = {Good Data Are Not Enough},
author = {Loeb, Avi},
date = {2016-11},
journaltitle = {Nature},
volume = {539},
pages = {23--25},
issn = {0028-0836},
doi = {10.1038/539023a},
url = {http://mfkp.org/INRMM/article/14176565},
abstract = {A vibrant scientific culture encourages many interpretations of evidence, argues Avi Loeb.
[Excerpt] [...] Most research funding is allocated assuming that the highest-quality data will inevitably deliver useful scientific interpretation and theoretical concepts, which can be tested and refined by future data. [...] To truly move forward, free thought must be encouraged outside the mainstream. Multiple interpretations of existing data and alternative motivations for collecting new data must be supported. [...]
[Blinkered view]
Mayan cosmologists had high social status. They got generous support because of their promises to forecast the future. Cosmologists today collect vast amounts of exquisite data in surveys of large parts of the sky, costing billions of dollars.
[] Such projects have a narrow aim -- pinning down the parameters of one theoretical model. The model comprises an expanding Universe composed of dark matter, dark energy and normal matter (from which stars, planets and people are made), with initial conditions dictated by an early phase of rapid expansion called cosmic inflation. The data are reduced to a few numbers. Surprises in the rest are tossed away.
[] I noticed this bias recently while assessing a PhD thesis. The student was asked to test whether a data set from a large cosmological survey was in line with the standard cosmological model. But when a discrepancy was found, the student's goal shifted to explaining why the data set was incomplete. In such a culture, the current model can never be ruled out, even though everyone knows that its major constituents [...] are not understood at a fundamental level.
[] Instead, observers should present results in a theory-neutral way. [...] Vast data sets may contain evidence for unusual behaviour that was unanticipated when the projects were conceived. If all results are expected and planned for, babies may be thrown out with the bathwater. [...]
[Alternative paths]
The only way to work out whether we are on the wrong path is to encourage competing interpretations of the known data.
[] I have been arguing for many years that funding agencies should promote the analysis of data for serendipitous purposes beyond major programmes and the mainstream dogma. [...] At a minimum, when funding is tight, a research frontier should maintain at least two ways of interpreting data so that new experiments will aim to select the correct one. [...] A healthy dialogue between different points of view should be fostered through multidisciplinary conferences that discuss conceptual issues, not just experimental results and phenomenology. A diversity of views fosters healthy progress and prevents stagnation. [...]},
keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-14176565,~to-add-doi-URL,cognitive-biases,confirmation-bias,cross-disciplinary-perspective,disciplinary-barrier,epistemology,research-management,transdisciplinary-research},
number = {7627}
}
Downloads: 0
{"_id":"kkgBM7FrvzG959KCQ","bibbaseid":"loeb-gooddataarenotenough","authorIDs":[],"author_short":["Loeb, A."],"bibdata":{"bibtype":"article","type":"article","title":"Good Data Are Not Enough","author":[{"propositions":[],"lastnames":["Loeb"],"firstnames":["Avi"],"suffixes":[]}],"date":"2016-11","journaltitle":"Nature","volume":"539","pages":"23–25","issn":"0028-0836","doi":"10.1038/539023a","url":"http://mfkp.org/INRMM/article/14176565","abstract":"A vibrant scientific culture encourages many interpretations of evidence, argues Avi Loeb. [Excerpt] [...] Most research funding is allocated assuming that the highest-quality data will inevitably deliver useful scientific interpretation and theoretical concepts, which can be tested and refined by future data. [...] To truly move forward, free thought must be encouraged outside the mainstream. Multiple interpretations of existing data and alternative motivations for collecting new data must be supported. [...] [Blinkered view] Mayan cosmologists had high social status. They got generous support because of their promises to forecast the future. Cosmologists today collect vast amounts of exquisite data in surveys of large parts of the sky, costing billions of dollars. [] Such projects have a narrow aim – pinning down the parameters of one theoretical model. The model comprises an expanding Universe composed of dark matter, dark energy and normal matter (from which stars, planets and people are made), with initial conditions dictated by an early phase of rapid expansion called cosmic inflation. The data are reduced to a few numbers. Surprises in the rest are tossed away. [] I noticed this bias recently while assessing a PhD thesis. The student was asked to test whether a data set from a large cosmological survey was in line with the standard cosmological model. But when a discrepancy was found, the student's goal shifted to explaining why the data set was incomplete. In such a culture, the current model can never be ruled out, even though everyone knows that its major constituents [...] are not understood at a fundamental level. [] Instead, observers should present results in a theory-neutral way. [...] Vast data sets may contain evidence for unusual behaviour that was unanticipated when the projects were conceived. If all results are expected and planned for, babies may be thrown out with the bathwater. [...] [Alternative paths] The only way to work out whether we are on the wrong path is to encourage competing interpretations of the known data. [] I have been arguing for many years that funding agencies should promote the analysis of data for serendipitous purposes beyond major programmes and the mainstream dogma. [...] At a minimum, when funding is tight, a research frontier should maintain at least two ways of interpreting data so that new experiments will aim to select the correct one. [...] A healthy dialogue between different points of view should be fostered through multidisciplinary conferences that discuss conceptual issues, not just experimental results and phenomenology. A diversity of views fosters healthy progress and prevents stagnation. [...]","keywords":"*imported-from-citeulike-INRMM,~INRMM-MiD:c-14176565,~to-add-doi-URL,cognitive-biases,confirmation-bias,cross-disciplinary-perspective,disciplinary-barrier,epistemology,research-management,transdisciplinary-research","number":"7627","bibtex":"@article{loebGoodDataAre2016,\n title = {Good Data Are Not Enough},\n author = {Loeb, Avi},\n date = {2016-11},\n journaltitle = {Nature},\n volume = {539},\n pages = {23--25},\n issn = {0028-0836},\n doi = {10.1038/539023a},\n url = {http://mfkp.org/INRMM/article/14176565},\n abstract = {A vibrant scientific culture encourages many interpretations of evidence, argues Avi Loeb.\n\n[Excerpt] [...] Most research funding is allocated assuming that the highest-quality data will inevitably deliver useful scientific interpretation and theoretical concepts, which can be tested and refined by future data. [...] To truly move forward, free thought must be encouraged outside the mainstream. Multiple interpretations of existing data and alternative motivations for collecting new data must be supported. [...] \n\n[Blinkered view]\n\nMayan cosmologists had high social status. They got generous support because of their promises to forecast the future. Cosmologists today collect vast amounts of exquisite data in surveys of large parts of the sky, costing billions of dollars.\n\n[] Such projects have a narrow aim -- pinning down the parameters of one theoretical model. The model comprises an expanding Universe composed of dark matter, dark energy and normal matter (from which stars, planets and people are made), with initial conditions dictated by an early phase of rapid expansion called cosmic inflation. The data are reduced to a few numbers. Surprises in the rest are tossed away.\n\n[] I noticed this bias recently while assessing a PhD thesis. The student was asked to test whether a data set from a large cosmological survey was in line with the standard cosmological model. But when a discrepancy was found, the student's goal shifted to explaining why the data set was incomplete. In such a culture, the current model can never be ruled out, even though everyone knows that its major constituents [...] are not understood at a fundamental level.\n\n[] Instead, observers should present results in a theory-neutral way. [...] Vast data sets may contain evidence for unusual behaviour that was unanticipated when the projects were conceived. If all results are expected and planned for, babies may be thrown out with the bathwater. [...]\n\n[Alternative paths]\n\nThe only way to work out whether we are on the wrong path is to encourage competing interpretations of the known data.\n\n[] I have been arguing for many years that funding agencies should promote the analysis of data for serendipitous purposes beyond major programmes and the mainstream dogma. [...] At a minimum, when funding is tight, a research frontier should maintain at least two ways of interpreting data so that new experiments will aim to select the correct one. [...] A healthy dialogue between different points of view should be fostered through multidisciplinary conferences that discuss conceptual issues, not just experimental results and phenomenology. A diversity of views fosters healthy progress and prevents stagnation. [...]},\n keywords = {*imported-from-citeulike-INRMM,~INRMM-MiD:c-14176565,~to-add-doi-URL,cognitive-biases,confirmation-bias,cross-disciplinary-perspective,disciplinary-barrier,epistemology,research-management,transdisciplinary-research},\n number = {7627}\n}\n\n","author_short":["Loeb, A."],"key":"loebGoodDataAre2016","id":"loebGoodDataAre2016","bibbaseid":"loeb-gooddataarenotenough","role":"author","urls":{"Paper":"http://mfkp.org/INRMM/article/14176565"},"keyword":["*imported-from-citeulike-INRMM","~INRMM-MiD:c-14176565","~to-add-doi-URL","cognitive-biases","confirmation-bias","cross-disciplinary-perspective","disciplinary-barrier","epistemology","research-management","transdisciplinary-research"],"downloads":0},"bibtype":"article","biburl":"https://tmpfiles.org/dl/58794/INRMM.bib","creationDate":"2020-07-02T22:41:12.787Z","downloads":0,"keywords":["*imported-from-citeulike-inrmm","~inrmm-mid:c-14176565","~to-add-doi-url","cognitive-biases","confirmation-bias","cross-disciplinary-perspective","disciplinary-barrier","epistemology","research-management","transdisciplinary-research"],"search_terms":["good","data","enough","loeb"],"title":"Good Data Are Not Enough","year":null,"dataSources":["DXuKbcZTirdigFKPF"]}