Data Distillation: Towards Omni-Supervised Learning. Radosavovic, I., Dollár, P., Girshick, R., Gkioxari, G., & He, K. December, 2017. arXiv:1712.04440 [cs]
Data Distillation: Towards Omni-Supervised Learning [link]Paper  doi  abstract   bibtex   
We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.
@misc{radosavovic_data_2017,
	title = {Data {Distillation}: {Towards} {Omni}-{Supervised} {Learning}},
	shorttitle = {Data {Distillation}},
	url = {http://arxiv.org/abs/1712.04440},
	doi = {10.48550/arXiv.1712.04440},
	abstract = {We investigate omni-supervised learning, a special regime of semi-supervised learning in which the learner exploits all available labeled data plus internet-scale sources of unlabeled data. Omni-supervised learning is lower-bounded by performance on existing labeled datasets, offering the potential to surpass state-of-the-art fully supervised methods. To exploit the omni-supervised setting, we propose data distillation, a method that ensembles predictions from multiple transformations of unlabeled data, using a single model, to automatically generate new training annotations. We argue that visual recognition models have recently become accurate enough that it is now possible to apply classic ideas about self-training to challenging real-world data. Our experimental results show that in the cases of human keypoint detection and general object detection, state-of-the-art models trained with data distillation surpass the performance of using labeled data from the COCO dataset alone.},
	language = {en},
	urldate = {2023-08-09},
	publisher = {arXiv},
	author = {Radosavovic, Ilija and Dollár, Piotr and Girshick, Ross and Gkioxari, Georgia and He, Kaiming},
	month = dec,
	year = {2017},
	note = {arXiv:1712.04440 [cs]},
	keywords = {\#CVPR{\textgreater}18, \#Deep Learning, \#Distilling, /unread, Computer Science - Computer Vision and Pattern Recognition},
}

Downloads: 0