Exponential Concentration of a Density Functional Estimator

Exponential Concentration of a Density Functional Estimator. Singh, S. & Poczos, B. In Advances in Neural Information Processing Systems, volume 27, 2014. Curran Associates, Inc..

Paper abstract bibtex

We analyse a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the d-dimensional unit cube [0,1]\textasciicircumd that lie in a beta-Holder smoothness class, we prove our estimator converges at the rate O(n\textasciicircum(1/(beta+d))). Furthermore, we prove that the estimator obeys an exponential concentration inequality about its mean, whereas most previous related results have bounded only expected error of estimators. Finally, we demonstrate our bounds to the case of conditional Renyi mutual information.

@inproceedings{singh_exponential_2014,
	title = {Exponential {Concentration} of a {Density} {Functional} {Estimator}},
	volume = {27},
	url = {https://proceedings.neurips.cc/paper/2014/hash/af5afd7f7c807171981d443ad4f4f648-Abstract.html},
	abstract = {We analyse a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the d-dimensional unit cube [0,1]{\textasciicircum}d that lie in a beta-Holder smoothness class, we prove our estimator converges at the rate O(n{\textasciicircum}(1/(beta+d))). Furthermore, we prove that the estimator obeys an exponential concentration inequality about its mean, whereas most previous related results have bounded only expected error of estimators. Finally, we demonstrate our bounds to the case of conditional Renyi mutual information.},
	urldate = {2023-03-04},
	booktitle = {Advances in {Neural} {Information} {Processing} {Systems}},
	publisher = {Curran Associates, Inc.},
	author = {Singh, Shashank and Poczos, Barnabas},
	year = {2014},
	file = {Full Text PDF:/Users/soumikp/Zotero/storage/LWALN2BZ/Singh and Poczos - 2014 - Exponential Concentration of a Density Functional .pdf:application/pdf},
}

Downloads: 0

{"_id":"W4LSiNoSrmDuxDurm","bibbaseid":"singh-poczos-exponentialconcentrationofadensityfunctionalestimator-2014","author_short":["Singh, S.","Poczos, B."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"Exponential Concentration of a Density Functional Estimator","volume":"27","url":"https://proceedings.neurips.cc/paper/2014/hash/af5afd7f7c807171981d443ad4f4f648-Abstract.html","abstract":"We analyse a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the d-dimensional unit cube [0,1]\\textasciicircumd that lie in a beta-Holder smoothness class, we prove our estimator converges at the rate O(n\\textasciicircum(1/(beta+d))). Furthermore, we prove that the estimator obeys an exponential concentration inequality about its mean, whereas most previous related results have bounded only expected error of estimators. Finally, we demonstrate our bounds to the case of conditional Renyi mutual information.","urldate":"2023-03-04","booktitle":"Advances in Neural Information Processing Systems","publisher":"Curran Associates, Inc.","author":[{"propositions":[],"lastnames":["Singh"],"firstnames":["Shashank"],"suffixes":[]},{"propositions":[],"lastnames":["Poczos"],"firstnames":["Barnabas"],"suffixes":[]}],"year":"2014","file":"Full Text PDF:/Users/soumikp/Zotero/storage/LWALN2BZ/Singh and Poczos - 2014 - Exponential Concentration of a Density Functional .pdf:application/pdf","bibtex":"@inproceedings{singh_exponential_2014,\n\ttitle = {Exponential {Concentration} of a {Density} {Functional} {Estimator}},\n\tvolume = {27},\n\turl = {https://proceedings.neurips.cc/paper/2014/hash/af5afd7f7c807171981d443ad4f4f648-Abstract.html},\n\tabstract = {We analyse a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the d-dimensional unit cube [0,1]{\\textasciicircum}d that lie in a beta-Holder smoothness class, we prove our estimator converges at the rate O(n{\\textasciicircum}(1/(beta+d))). Furthermore, we prove that the estimator obeys an exponential concentration inequality about its mean, whereas most previous related results have bounded only expected error of estimators. Finally, we demonstrate our bounds to the case of conditional Renyi mutual information.},\n\turldate = {2023-03-04},\n\tbooktitle = {Advances in {Neural} {Information} {Processing} {Systems}},\n\tpublisher = {Curran Associates, Inc.},\n\tauthor = {Singh, Shashank and Poczos, Barnabas},\n\tyear = {2014},\n\tfile = {Full Text PDF:/Users/soumikp/Zotero/storage/LWALN2BZ/Singh and Poczos - 2014 - Exponential Concentration of a Density Functional .pdf:application/pdf},\n}\n\n","author_short":["Singh, S.","Poczos, B."],"key":"singh_exponential_2014","id":"singh_exponential_2014","bibbaseid":"singh-poczos-exponentialconcentrationofadensityfunctionalestimator-2014","role":"author","urls":{"Paper":"https://proceedings.neurips.cc/paper/2014/hash/af5afd7f7c807171981d443ad4f4f648-Abstract.html"},"metadata":{"authorlinks":{}},"html":""},"bibtype":"inproceedings","biburl":"https://bibbase.org/f/Ceciz2iNjTZgQNtDc/mypubs_mar_2024.bib","dataSources":["m8Y57GfgnRrMKZTQS","epk5yKhDyD37NAsSC","93mP5kMGHQy5bo5xJ"],"keywords":[],"search_terms":["exponential","concentration","density","functional","estimator","singh","poczos"],"title":"Exponential Concentration of a Density Functional Estimator","year":2014}