Improving Accountability in Recommender Systems Research Through Reproducibility. Bellogín, A. & Said, A. January 2021. ISBN: 2102.00482 Publication Title: arXiv [cs.IR]![Improving Accountability in Recommender Systems Research Through Reproducibility [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper abstract bibtex Reproducibility is a key requirement for scientific progress. It allows the reproduction of the works of others, and, as a consequence, to fully trust the reported claims and results. In this work, we argue that, by facilitating reproducibility of recommender systems experimentation, we indirectly address the issues of accountability and transparency in recommender systems research from the perspectives of practitioners, designers, and engineers aiming to assess the capabilities of published research works. These issues have become increasingly prevalent in recent literature. Reasons for this include societal movements around intelligent systems and artificial intelligence striving towards fair and objective use of human behavioral data (as in Machine Learning, Information Retrieval, or Human-Computer Interaction). Society has grown to expect explanations and transparency standards regarding the underlying algorithms making automated decisions for and around us. This work surveys existing definitions of these concepts, and proposes a coherent terminology for recommender systems research, with the goal to connect reproducibility to accountability. We achieve this by introducing several guidelines and steps that lead to reproducible and, hence, accountable experimental workflows and research. We additionally analyze several instantiations of recommender system implementations available in the literature, and discuss the extent to which they fit in the introduced framework. With this work, we aim to shed light on this important problem, and facilitate progress in the field by increasing the accountability of research.
@unpublished{bellogin_improving_2021,
title = {Improving {Accountability} in {Recommender} {Systems} {Research} {Through} {Reproducibility}},
url = {http://arxiv.org/abs/2102.00482},
abstract = {Reproducibility is a key requirement for scientific progress. It allows
the reproduction of the works of others, and, as a consequence, to fully
trust the reported claims and results. In this work, we argue that, by
facilitating reproducibility of recommender systems experimentation, we
indirectly address the issues of accountability and transparency in
recommender systems research from the perspectives of practitioners,
designers, and engineers aiming to assess the capabilities of published
research works. These issues have become increasingly prevalent in recent
literature. Reasons for this include societal movements around intelligent
systems and artificial intelligence striving towards fair and objective
use of human behavioral data (as in Machine Learning, Information
Retrieval, or Human-Computer Interaction). Society has grown to expect
explanations and transparency standards regarding the underlying
algorithms making automated decisions for and around us. This work surveys
existing definitions of these concepts, and proposes a coherent
terminology for recommender systems research, with the goal to connect
reproducibility to accountability. We achieve this by introducing several
guidelines and steps that lead to reproducible and, hence, accountable
experimental workflows and research. We additionally analyze several
instantiations of recommender system implementations available in the
literature, and discuss the extent to which they fit in the introduced
framework. With this work, we aim to shed light on this important problem,
and facilitate progress in the field by increasing the accountability of
research.},
author = {Bellogín, Alejandro and Said, Alan},
month = jan,
year = {2021},
note = {ISBN: 2102.00482
Publication Title: arXiv [cs.IR]},
}
Downloads: 0
{"_id":"HHzDhv3iwvq9xgJcT","bibbaseid":"bellogn-said-improvingaccountabilityinrecommendersystemsresearchthroughreproducibility-2021","author_short":["Bellogín, A.","Said, A."],"bibdata":{"bibtype":"unpublished","type":"unpublished","title":"Improving Accountability in Recommender Systems Research Through Reproducibility","url":"http://arxiv.org/abs/2102.00482","abstract":"Reproducibility is a key requirement for scientific progress. It allows the reproduction of the works of others, and, as a consequence, to fully trust the reported claims and results. In this work, we argue that, by facilitating reproducibility of recommender systems experimentation, we indirectly address the issues of accountability and transparency in recommender systems research from the perspectives of practitioners, designers, and engineers aiming to assess the capabilities of published research works. These issues have become increasingly prevalent in recent literature. Reasons for this include societal movements around intelligent systems and artificial intelligence striving towards fair and objective use of human behavioral data (as in Machine Learning, Information Retrieval, or Human-Computer Interaction). Society has grown to expect explanations and transparency standards regarding the underlying algorithms making automated decisions for and around us. This work surveys existing definitions of these concepts, and proposes a coherent terminology for recommender systems research, with the goal to connect reproducibility to accountability. We achieve this by introducing several guidelines and steps that lead to reproducible and, hence, accountable experimental workflows and research. We additionally analyze several instantiations of recommender system implementations available in the literature, and discuss the extent to which they fit in the introduced framework. With this work, we aim to shed light on this important problem, and facilitate progress in the field by increasing the accountability of research.","author":[{"propositions":[],"lastnames":["Bellogín"],"firstnames":["Alejandro"],"suffixes":[]},{"propositions":[],"lastnames":["Said"],"firstnames":["Alan"],"suffixes":[]}],"month":"January","year":"2021","note":"ISBN: 2102.00482 Publication Title: arXiv [cs.IR]","bibtex":"@unpublished{bellogin_improving_2021,\n\ttitle = {Improving {Accountability} in {Recommender} {Systems} {Research} {Through} {Reproducibility}},\n\turl = {http://arxiv.org/abs/2102.00482},\n\tabstract = {Reproducibility is a key requirement for scientific progress. It allows\nthe reproduction of the works of others, and, as a consequence, to fully\ntrust the reported claims and results. In this work, we argue that, by\nfacilitating reproducibility of recommender systems experimentation, we\nindirectly address the issues of accountability and transparency in\nrecommender systems research from the perspectives of practitioners,\ndesigners, and engineers aiming to assess the capabilities of published\nresearch works. These issues have become increasingly prevalent in recent\nliterature. Reasons for this include societal movements around intelligent\nsystems and artificial intelligence striving towards fair and objective\nuse of human behavioral data (as in Machine Learning, Information\nRetrieval, or Human-Computer Interaction). Society has grown to expect\nexplanations and transparency standards regarding the underlying\nalgorithms making automated decisions for and around us. This work surveys\nexisting definitions of these concepts, and proposes a coherent\nterminology for recommender systems research, with the goal to connect\nreproducibility to accountability. We achieve this by introducing several\nguidelines and steps that lead to reproducible and, hence, accountable\nexperimental workflows and research. We additionally analyze several\ninstantiations of recommender system implementations available in the\nliterature, and discuss the extent to which they fit in the introduced\nframework. With this work, we aim to shed light on this important problem,\nand facilitate progress in the field by increasing the accountability of\nresearch.},\n\tauthor = {Bellogín, Alejandro and Said, Alan},\n\tmonth = jan,\n\tyear = {2021},\n\tnote = {ISBN: 2102.00482\nPublication Title: arXiv [cs.IR]},\n}\n\n","author_short":["Bellogín, A.","Said, A."],"key":"bellogin_improving_2021","id":"bellogin_improving_2021","bibbaseid":"bellogn-said-improvingaccountabilityinrecommendersystemsresearchthroughreproducibility-2021","role":"author","urls":{"Paper":"http://arxiv.org/abs/2102.00482"},"metadata":{"authorlinks":{}}},"bibtype":"unpublished","biburl":"https://api.zotero.org/users/6655/collections/TJPPJ92X/items?key=VFvZhZXIoHNBbzoLZ1IM2zgf&format=bibtex&limit=100","dataSources":["5EkadiRxQiXibn7T2","jfoasiDDpStqkkoZB","BiuuFc45aHCgJqDLY"],"keywords":[],"search_terms":["improving","accountability","recommender","systems","research","through","reproducibility","bellogín","said"],"title":"Improving Accountability in Recommender Systems Research Through Reproducibility","year":2021}