Debunking Generalization Error or: How I Learned to Stop Worrying and Love My Training Set. Acquaviva, V., Lovell, C., & Ishida, E. NeurIPS workshop `Machine Learning and the Physical Sciences', November, 2020. ![Debunking Generalization Error or: How I Learned to Stop Worrying and Love My Training Set [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper abstract bibtex We aim to determine some physical properties of distant galaxies (for example, stellar mass, star formation history, or chemical enrichment history) from their observed spectra, using supervised machine learning methods. We know that different astrophysical processes leave their imprint in various regions of the spectra with characteristic signatures. Unfortunately, identifying a training set for this problem is very hard, because labels are not readily available - we have no way of knowing the true history of how galaxies have formed. One possible approach to this problem is to train machine learning models on state-of-the-art cosmological simulations. However, when algorithms are trained on the simulations, it is unclear how well they will perform once applied to real data. In this paper, we attempt to model the generalization error as a function of an appropriate measure of distance between the source domain and the application domain. Our goal is to obtain a reliable estimate of how a model trained on simulations might behave on data.
@article{acquaviva_debunking_2020,
title = {Debunking {Generalization} {Error} or: {How} {I} {Learned} to {Stop} {Worrying} and {Love} {My} {Training} {Set}},
shorttitle = {Debunking {Generalization} {Error}},
url = {http://adsabs.harvard.edu/abs/2020arXiv201200066A},
abstract = {We aim to determine some physical properties of distant galaxies (for
example, stellar mass, star formation history, or chemical enrichment
history) from their observed spectra, using supervised machine learning
methods. We know that different astrophysical processes leave their
imprint in various regions of the spectra with characteristic
signatures. Unfortunately, identifying a training set for this problem
is very hard, because labels are not readily available - we have no way
of knowing the true history of how galaxies have formed. One possible
approach to this problem is to train machine learning models on
state-of-the-art cosmological simulations. However, when algorithms are
trained on the simulations, it is unclear how well they will perform
once applied to real data. In this paper, we attempt to model the
generalization error as a function of an appropriate measure of distance
between the source domain and the application domain. Our goal is to
obtain a reliable estimate of how a model trained on simulations might
behave on data.},
urldate = {2020-12-02},
journal = {NeurIPS workshop `Machine Learning and the Physical Sciences'},
author = {Acquaviva, Viviana and Lovell, Chistopher and Ishida, Emille},
month = nov,
year = {2020},
keywords = {Astrophysics - Astrophysics of Galaxies, Astrophysics - Instrumentation and Methods for Astrophysics},
}
Downloads: 0
{"_id":"fwGJR8i5WcXQLRrYk","bibbaseid":"acquaviva-lovell-ishida-debunkinggeneralizationerrororhowilearnedtostopworryingandlovemytrainingset-2020","author_short":["Acquaviva, V.","Lovell, C.","Ishida, E."],"bibdata":{"bibtype":"article","type":"article","title":"Debunking Generalization Error or: How I Learned to Stop Worrying and Love My Training Set","shorttitle":"Debunking Generalization Error","url":"http://adsabs.harvard.edu/abs/2020arXiv201200066A","abstract":"We aim to determine some physical properties of distant galaxies (for example, stellar mass, star formation history, or chemical enrichment history) from their observed spectra, using supervised machine learning methods. We know that different astrophysical processes leave their imprint in various regions of the spectra with characteristic signatures. Unfortunately, identifying a training set for this problem is very hard, because labels are not readily available - we have no way of knowing the true history of how galaxies have formed. One possible approach to this problem is to train machine learning models on state-of-the-art cosmological simulations. However, when algorithms are trained on the simulations, it is unclear how well they will perform once applied to real data. In this paper, we attempt to model the generalization error as a function of an appropriate measure of distance between the source domain and the application domain. Our goal is to obtain a reliable estimate of how a model trained on simulations might behave on data.","urldate":"2020-12-02","journal":"NeurIPS workshop `Machine Learning and the Physical Sciences'","author":[{"propositions":[],"lastnames":["Acquaviva"],"firstnames":["Viviana"],"suffixes":[]},{"propositions":[],"lastnames":["Lovell"],"firstnames":["Chistopher"],"suffixes":[]},{"propositions":[],"lastnames":["Ishida"],"firstnames":["Emille"],"suffixes":[]}],"month":"November","year":"2020","keywords":"Astrophysics - Astrophysics of Galaxies, Astrophysics - Instrumentation and Methods for Astrophysics","bibtex":"@article{acquaviva_debunking_2020,\n\ttitle = {Debunking {Generalization} {Error} or: {How} {I} {Learned} to {Stop} {Worrying} and {Love} {My} {Training} {Set}},\n\tshorttitle = {Debunking {Generalization} {Error}},\n\turl = {http://adsabs.harvard.edu/abs/2020arXiv201200066A},\n\tabstract = {We aim to determine some physical properties of distant galaxies (for \nexample, stellar mass, star formation history, or chemical enrichment\nhistory) from their observed spectra, using supervised machine learning\nmethods. We know that different astrophysical processes leave their\nimprint in various regions of the spectra with characteristic\nsignatures. Unfortunately, identifying a training set for this problem\nis very hard, because labels are not readily available - we have no way\nof knowing the true history of how galaxies have formed. One possible\napproach to this problem is to train machine learning models on\nstate-of-the-art cosmological simulations. However, when algorithms are\ntrained on the simulations, it is unclear how well they will perform\nonce applied to real data. In this paper, we attempt to model the\ngeneralization error as a function of an appropriate measure of distance\nbetween the source domain and the application domain. Our goal is to\nobtain a reliable estimate of how a model trained on simulations might\nbehave on data.},\n\turldate = {2020-12-02},\n\tjournal = {NeurIPS workshop `Machine Learning and the Physical Sciences'},\n\tauthor = {Acquaviva, Viviana and Lovell, Chistopher and Ishida, Emille},\n\tmonth = nov,\n\tyear = {2020},\n\tkeywords = {Astrophysics - Astrophysics of Galaxies, Astrophysics - Instrumentation and Methods for Astrophysics},\n}\n\n","author_short":["Acquaviva, V.","Lovell, C.","Ishida, E."],"key":"acquaviva_debunking_2020","id":"acquaviva_debunking_2020","bibbaseid":"acquaviva-lovell-ishida-debunkinggeneralizationerrororhowilearnedtostopworryingandlovemytrainingset-2020","role":"author","urls":{"Paper":"http://adsabs.harvard.edu/abs/2020arXiv201200066A"},"keyword":["Astrophysics - Astrophysics of Galaxies","Astrophysics - Instrumentation and Methods for Astrophysics"],"metadata":{"authorlinks":{}}},"bibtype":"article","biburl":"https://api.zotero.org/users/2650003/collections/2GYAYTB9/items?key=HjJ9T6LSEhBUjY15bMA811su&format=bibtex&limit=100","dataSources":["7gvjSdWrEu7z5vjjj","CxJmmGJKTkqa9B6aM"],"keywords":["astrophysics - astrophysics of galaxies","astrophysics - instrumentation and methods for astrophysics"],"search_terms":["debunking","generalization","error","learned","stop","worrying","love","training","set","acquaviva","lovell","ishida"],"title":"Debunking Generalization Error or: How I Learned to Stop Worrying and Love My Training Set","year":2020}