PAC-Bayesian Contrastive Unsupervised Representation Learning. Nozawa, K., Germain, P., & Guedj, B. In Conference on Uncertainty in Artificial Intelligence [UAI], 2020.
![PAC-Bayesian Contrastive Unsupervised Representation Learning [link]](https://bibbase.org/img/filetypes/link.svg "link")
Paper Arxiv ![PAC-Bayesian Contrastive Unsupervised Representation Learning [pdf]](https://bibbase.org/img/filetypes/pdf.svg "pdf")
Pdf Supplementary Code Video abstract bibtex 1 download
Paper Arxiv Pdf Supplementary Code Video abstract bibtex 1 download Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.
@inproceedings{nozawa2019pacbayesian,
title={{PAC-Bayesian} Contrastive Unsupervised Representation Learning},
author={Kento Nozawa and Pascal Germain and Benjamin Guedj},
year={2020},
abstract = "Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.",
booktitle = {{Conference on Uncertainty in Artificial Intelligence [UAI]}},
url = "https://proceedings.mlr.press/v124/nozawa20a.html",
url_arXiv = "https://arxiv.org/abs/1910.04464",
url_PDF = "http://proceedings.mlr.press/v124/nozawa20a/nozawa20a.pdf",
url_Supplementary = "http://proceedings.mlr.press/v124/nozawa20a/nozawa20a-supp.pdf",
url_Code = "https://github.com/nzw0301/pb-contrastive",
url_Video = "https://youtu.be/WUh3Fgo5nhY",
eprint={1910.04464},
archivePrefix={arXiv},
primaryClass={cs.LG},
keywords={mine}
}Downloads: 1
{"_id":"24nZL2TbHdmLP5hBu","bibbaseid":"nozawa-germain-guedj-pacbayesiancontrastiveunsupervisedrepresentationlearning-2020","author_short":["Nozawa, K.","Germain, P.","Guedj, B."],"bibdata":{"bibtype":"inproceedings","type":"inproceedings","title":"PAC-Bayesian Contrastive Unsupervised Representation Learning","author":[{"firstnames":["Kento"],"propositions":[],"lastnames":["Nozawa"],"suffixes":[]},{"firstnames":["Pascal"],"propositions":[],"lastnames":["Germain"],"suffixes":[]},{"firstnames":["Benjamin"],"propositions":[],"lastnames":["Guedj"],"suffixes":[]}],"year":"2020","abstract":"Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.","booktitle":"Conference on Uncertainty in Artificial Intelligence [UAI]","url":"https://proceedings.mlr.press/v124/nozawa20a.html","url_arxiv":"https://arxiv.org/abs/1910.04464","url_pdf":"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a.pdf","url_supplementary":"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a-supp.pdf","url_code":"https://github.com/nzw0301/pb-contrastive","url_video":"https://youtu.be/WUh3Fgo5nhY","eprint":"1910.04464","archiveprefix":"arXiv","primaryclass":"cs.LG","keywords":"mine","bibtex":"@inproceedings{nozawa2019pacbayesian,\ntitle={{PAC-Bayesian} Contrastive Unsupervised Representation Learning},\nauthor={Kento Nozawa and Pascal Germain and Benjamin Guedj},\nyear={2020},\nabstract = \"Contrastive unsupervised representation learning (CURL) is the state-of-the-art technique to learn representations (as a set of features) from unlabelled data. While CURL has collected several empirical successes recently, theoretical understanding of its performance was still missing. In a recent work, Arora et al. (2019) provide the first generalisation bounds for CURL, relying on a Rademacher complexity. We extend their framework to the flexible PAC-Bayes setting, allowing us to deal with the non-iid setting. We present PAC-Bayesian generalisation bounds for CURL, which are then used to derive a new representation learning algorithm. Numerical experiments on real-life datasets illustrate that our algorithm achieves competitive accuracy, and yields non-vacuous generalisation bounds.\",\nbooktitle = {{Conference on Uncertainty in Artificial Intelligence [UAI]}},\nurl = \"https://proceedings.mlr.press/v124/nozawa20a.html\",\nurl_arXiv = \"https://arxiv.org/abs/1910.04464\",\nurl_PDF = \"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a.pdf\",\nurl_Supplementary = \"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a-supp.pdf\",\nurl_Code = \"https://github.com/nzw0301/pb-contrastive\",\nurl_Video = \"https://youtu.be/WUh3Fgo5nhY\",\neprint={1910.04464},\narchivePrefix={arXiv},\nprimaryClass={cs.LG},\nkeywords={mine}\n}\n\n","author_short":["Nozawa, K.","Germain, P.","Guedj, B."],"key":"nozawa2019pacbayesian","id":"nozawa2019pacbayesian","bibbaseid":"nozawa-germain-guedj-pacbayesiancontrastiveunsupervisedrepresentationlearning-2020","role":"author","urls":{"Paper":"https://proceedings.mlr.press/v124/nozawa20a.html"," arxiv":"https://arxiv.org/abs/1910.04464"," pdf":"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a.pdf"," supplementary":"http://proceedings.mlr.press/v124/nozawa20a/nozawa20a-supp.pdf"," code":"https://github.com/nzw0301/pb-contrastive"," video":"https://youtu.be/WUh3Fgo5nhY"},"keyword":["mine"],"metadata":{"authorlinks":{}},"downloads":1,"html":""},"bibtype":"inproceedings","biburl":"https://bguedj.github.io/files/bguedj-publications.bib","dataSources":["suE7RgYeZEnSYr5Fy"],"keywords":["mine"],"search_terms":["pac","bayesian","contrastive","unsupervised","representation","learning","nozawa","germain","guedj"],"title":"PAC-Bayesian Contrastive Unsupervised Representation Learning","year":2020,"downloads":1}