Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective. Smith, J. J., Beattie, L., & Cramer, H. In Proceedings of the ACM web conference 2023, pages 3648–3659, New York, NY, USA, April, 2023. Association for Computing Machinery. ![Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper doi abstract bibtex Measuring and assessing the impact and “fairness’’ of recommendation algorithms is central to responsible recommendation efforts. However, the complexity of fairness definitions and the proliferation of fairness metrics in research literature have led to a complex decision-making space. This environment makes it challenging for practitioners to operationalize and pick metrics that work within their unique context. This suggests that practitioners require more decision-making support, but it is not clear what type of support would be beneficial. We conducted a literature review of 24 papers to gather metrics introduced by the research community for measuring fairness in recommendation and ranking systems. We organized these metrics into a ‘decision-tree style’ support framework designed to help practitioners scope fairness objectives and identify fairness metrics relevant to their recommendation domain and application context. To explore the feasibility of this approach, we conducted 15 semi-structured interviews using this framework to assess which challenges practitioners may face when scoping fairness objectives and metrics for their system, and which further support may be needed beyond such tools.
@inproceedings{smith_scoping_2023,
address = {New York, NY, USA},
title = {Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective},
isbn = {978-1-4503-9416-1},
shorttitle = {Scoping fairness objectives and identifying fairness metrics for recommender systems},
url = {https://dl.acm.org/doi/10.1145/3543507.3583204},
doi = {10.1145/3543507.3583204},
abstract = {Measuring and assessing the impact and “fairness’’ of recommendation algorithms is central to responsible recommendation efforts. However, the complexity of fairness definitions and the proliferation of fairness metrics in research literature have led to a complex decision-making space. This environment makes it challenging for practitioners to operationalize and pick metrics that work within their unique context. This suggests that practitioners require more decision-making support, but it is not clear what type of support would be beneficial. We conducted a literature review of 24 papers to gather metrics introduced by the research community for measuring fairness in recommendation and ranking systems. We organized these metrics into a ‘decision-tree style’ support framework designed to help practitioners scope fairness objectives and identify fairness metrics relevant to their recommendation domain and application context. To explore the feasibility of this approach, we conducted 15 semi-structured interviews using this framework to assess which challenges practitioners may face when scoping fairness objectives and metrics for their system, and which further support may be needed beyond such tools.},
urldate = {2024-01-10},
booktitle = {Proceedings of the {ACM} web conference 2023},
publisher = {Association for Computing Machinery},
author = {Smith, Jessie J. and Beattie, Lex and Cramer, Henriette},
month = apr,
year = {2023},
keywords = {algorithmic audits, algorithmic bias, fairness, recommender systems},
pages = {3648--3659},
}
Downloads: 0
{"_id":"QuehMRiuxmvtQQBdb","bibbaseid":"smith-beattie-cramer-scopingfairnessobjectivesandidentifyingfairnessmetricsforrecommendersystemsthepractitionersperspective-2023","author_short":["Smith, J. J.","Beattie, L.","Cramer, H."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","address":"New York, NY, USA","title":"Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective","isbn":"978-1-4503-9416-1","shorttitle":"Scoping fairness objectives and identifying fairness metrics for recommender systems","url":"https://dl.acm.org/doi/10.1145/3543507.3583204","doi":"10.1145/3543507.3583204","abstract":"Measuring and assessing the impact and “fairness’’ of recommendation algorithms is central to responsible recommendation efforts. However, the complexity of fairness definitions and the proliferation of fairness metrics in research literature have led to a complex decision-making space. This environment makes it challenging for practitioners to operationalize and pick metrics that work within their unique context. This suggests that practitioners require more decision-making support, but it is not clear what type of support would be beneficial. We conducted a literature review of 24 papers to gather metrics introduced by the research community for measuring fairness in recommendation and ranking systems. We organized these metrics into a ‘decision-tree style’ support framework designed to help practitioners scope fairness objectives and identify fairness metrics relevant to their recommendation domain and application context. To explore the feasibility of this approach, we conducted 15 semi-structured interviews using this framework to assess which challenges practitioners may face when scoping fairness objectives and metrics for their system, and which further support may be needed beyond such tools.","urldate":"2024-01-10","booktitle":"Proceedings of the ACM web conference 2023","publisher":"Association for Computing Machinery","author":[{"propositions":[],"lastnames":["Smith"],"firstnames":["Jessie","J."],"suffixes":[]},{"propositions":[],"lastnames":["Beattie"],"firstnames":["Lex"],"suffixes":[]},{"propositions":[],"lastnames":["Cramer"],"firstnames":["Henriette"],"suffixes":[]}],"month":"April","year":"2023","keywords":"algorithmic audits, algorithmic bias, fairness, recommender systems","pages":"3648–3659","bibtex":"@inproceedings{smith_scoping_2023,\n\taddress = {New York, NY, USA},\n\ttitle = {Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective},\n\tisbn = {978-1-4503-9416-1},\n\tshorttitle = {Scoping fairness objectives and identifying fairness metrics for recommender systems},\n\turl = {https://dl.acm.org/doi/10.1145/3543507.3583204},\n\tdoi = {10.1145/3543507.3583204},\n\tabstract = {Measuring and assessing the impact and “fairness’’ of recommendation algorithms is central to responsible recommendation efforts. However, the complexity of fairness definitions and the proliferation of fairness metrics in research literature have led to a complex decision-making space. This environment makes it challenging for practitioners to operationalize and pick metrics that work within their unique context. This suggests that practitioners require more decision-making support, but it is not clear what type of support would be beneficial. We conducted a literature review of 24 papers to gather metrics introduced by the research community for measuring fairness in recommendation and ranking systems. We organized these metrics into a ‘decision-tree style’ support framework designed to help practitioners scope fairness objectives and identify fairness metrics relevant to their recommendation domain and application context. To explore the feasibility of this approach, we conducted 15 semi-structured interviews using this framework to assess which challenges practitioners may face when scoping fairness objectives and metrics for their system, and which further support may be needed beyond such tools.},\n\turldate = {2024-01-10},\n\tbooktitle = {Proceedings of the {ACM} web conference 2023},\n\tpublisher = {Association for Computing Machinery},\n\tauthor = {Smith, Jessie J. and Beattie, Lex and Cramer, Henriette},\n\tmonth = apr,\n\tyear = {2023},\n\tkeywords = {algorithmic audits, algorithmic bias, fairness, recommender systems},\n\tpages = {3648--3659},\n}\n\n","author_short":["Smith, J. J.","Beattie, L.","Cramer, H."],"key":"smith_scoping_2023","id":"smith_scoping_2023","bibbaseid":"smith-beattie-cramer-scopingfairnessobjectivesandidentifyingfairnessmetricsforrecommendersystemsthepractitionersperspective-2023","role":"author","urls":{"Paper":"https://dl.acm.org/doi/10.1145/3543507.3583204"},"keyword":["algorithmic audits","algorithmic bias","fairness","recommender systems"],"metadata":{"authorlinks":{}}},"bibtype":"inproceedings","biburl":"https://api.zotero.org/users/6655/collections/PXABRENG/items?key=f7dGmoR42f4Se5vp1eOrI4Kf&format=bibtex&limit=100","dataSources":["iXpvky2uc5Bb3jNYX"],"keywords":["algorithmic audits","algorithmic bias","fairness","recommender systems"],"search_terms":["scoping","fairness","objectives","identifying","fairness","metrics","recommender","systems","practitioners","perspective","smith","beattie","cramer"],"title":"Scoping fairness objectives and identifying fairness metrics for recommender systems: the practitioners’ perspective","year":2023}