In Balcan, M. F. & Weinberger, K. Q., editors, *Proceedings of The 33rd International Conference on Machine Learning*, volume 48, of *Proceedings of Machine Learning Research*, pages 304--313, New York, New York, USA, 20--22 Jun, 2016. PMLR.

Paper abstract bibtex

Paper abstract bibtex

Given samples from two densities p and q, density ratio estimation (DRE) is the problem of estimating the ratio p/q. Two popular discriminative approaches to DRE are KL importance estimation (KLIEP), and least squares importance fitting (LSIF). In this paper, we show that KLIEP and LSIF both employ class-probability estimation (CPE) losses. Motivated by this, we formally relate DRE and CPE, and demonstrate the viability of using existing losses from one problem for the other. For the DRE problem, we show that essentially any CPE loss (eg logistic, exponential) can be used, as this equivalently minimises a Bregman divergence to the true density ratio. We show how different losses focus on accurately modelling different ranges of the density ratio, and use this to design new CPE losses for DRE. For the CPE problem, we argue that the LSIF loss is useful in the regime where one wishes to rank instances with maximal accuracy at the head of the ranking. In the course of our analysis, we establish a Bregman divergence identity that may be of independent interest.

@inproceedings{pmlr-v48-menon16, Abstract = {Given samples from two densities p and q, density ratio estimation (DRE) is the problem of estimating the ratio p/q. Two popular discriminative approaches to DRE are KL importance estimation (KLIEP), and least squares importance fitting (LSIF). In this paper, we show that KLIEP and LSIF both employ class-probability estimation (CPE) losses. Motivated by this, we formally relate DRE and CPE, and demonstrate the viability of using existing losses from one problem for the other. For the DRE problem, we show that essentially any CPE loss (eg logistic, exponential) can be used, as this equivalently minimises a Bregman divergence to the true density ratio. We show how different losses focus on accurately modelling different ranges of the density ratio, and use this to design new CPE losses for DRE. For the CPE problem, we argue that the LSIF loss is useful in the regime where one wishes to rank instances with maximal accuracy at the head of the ranking. In the course of our analysis, we establish a Bregman divergence identity that may be of independent interest.}, Address = {New York, New York, USA}, Author = {Aditya Menon and Cheng Soon Ong}, Booktitle = {Proceedings of The 33rd International Conference on Machine Learning}, Date-Modified = {2017-09-27 08:52:57 +0000}, Editor = {Maria Florina Balcan and Kilian Q. Weinberger}, Keywords = {machine learning}, Month = {20--22 Jun}, Pages = {304--313}, Pdf = {http://proceedings.mlr.press/v48/menon16.pdf}, Publisher = {PMLR}, Series = {Proceedings of Machine Learning Research}, Title = {Linking losses for density ratio and class-probability estimation}, Url = {http://proceedings.mlr.press/v48/menon16.html}, Volume = {48}, Year = {2016}, Bdsk-Url-1 = {http://proceedings.mlr.press/v48/menon16.html}}

Downloads: 0