Fairness in Recommendation Ranking through Pairwise Comparisons. Beutel, A., Chen, J., Doshi, T., Qian, H., Wei, L., Wu, Y., Heldt, L., Zhao, Z., Hong, L., Chi, E. H., & Goodrow, C.
Fairness in Recommendation Ranking through Pairwise Comparisons [link]Paper  abstract   bibtex   
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information. As such it is important to ask: what are the possible fairness risks, how can we quantify them, and how should we address them? In this paper we offer a set of novel metrics for evaluating algorithmic fairness concerns in recommender systems. In particular we show how measuring fairness based on pairwise comparisons from randomized experiments provides a tractable means to reason about fairness in rankings from recommender systems. Building on this metric, we offer a new regularizer to encourage improving this metric during model training and thus improve fairness in the resulting rankings. We apply this pairwise regularization to a large-scale, production recommender system and show that we are able to significantly improve the system's pairwise fairness.
@article{beutel_fairness_2019,
	title = {Fairness in Recommendation Ranking through Pairwise Comparisons},
	url = {http://arxiv.org/abs/1903.00780},
	abstract = {Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information. As such it is important to ask: what are the possible fairness risks, how can we quantify them, and how should we address them? In this paper we offer a set of novel metrics for evaluating algorithmic fairness concerns in recommender systems. In particular we show how measuring fairness based on pairwise comparisons from randomized experiments provides a tractable means to reason about fairness in rankings from recommender systems. Building on this metric, we offer a new regularizer to encourage improving this metric during model training and thus improve fairness in the resulting rankings. We apply this pairwise regularization to a large-scale, production recommender system and show that we are able to significantly improve the system's pairwise fairness.},
	journaltitle = {{arXiv}:1903.00780 [cs, stat]},
	author = {Beutel, Alex and Chen, Jilin and Doshi, Tulsee and Qian, Hai and Wei, Li and Wu, Yi and Heldt, Lukasz and Zhao, Zhe and Hong, Lichan and Chi, Ed H. and Goodrow, Cristos},
	urldate = {2019-04-29},
	date = {2019-03-02},
	eprinttype = {arxiv},
	eprint = {1903.00780},
	keywords = {Computer Science - Information Retrieval, Computer Science - Machine Learning, Statistics - Machine Learning, Computer Science - Computers and Society, Computer Science - Artificial Intelligence}
}

Downloads: 0