CAFS: class adaptive framework for semi-supervised semantic segmentation

CAFS: class adaptive framework for semi-supervised semantic segmentation. Ju, J., Noh, H., Wang, Y., Seo, M., & Choi, D. March, 2023. arXiv:2303.11606 [cs]

Paper abstract bibtex

Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by selftraining with pseudo-labeling pixels having high confidences for unlabeled images. However, using only highconfidence pixels for self-training may result in losing much of the information in the unlabeled datasets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) to cope with the loss of most information that occurs in existing high-confidence-based pseudolabeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class. On this basis, we propose a calibration aware class-wise adaptive thresholding and classwise adaptive oversampling using the analysis results from the validation set. Our proposed CAFS achieves state-ofthe-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0% and 80.4%, respectively. The code is available at https://github.com/cjf8899/CAFS.

@misc{ju_cafs_2023,
	title = {{CAFS}: class adaptive framework for semi-supervised semantic segmentation},
	shorttitle = {Cafs},
	url = {http://arxiv.org/abs/2303.11606},
	abstract = {Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by selftraining with pseudo-labeling pixels having high confidences for unlabeled images. However, using only highconfidence pixels for self-training may result in losing much of the information in the unlabeled datasets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) to cope with the loss of most information that occurs in existing high-confidence-based pseudolabeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class. On this basis, we propose a calibration aware class-wise adaptive thresholding and classwise adaptive oversampling using the analysis results from the validation set. Our proposed CAFS achieves state-ofthe-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0\% and 80.4\%, respectively. The code is available at https://github.com/cjf8899/CAFS.},
	language = {en},
	urldate = {2023-05-10},
	publisher = {arXiv},
	author = {Ju, Jingi and Noh, Hyeoncheol and Wang, Yooseung and Seo, Minseok and Choi, Dong-Geol},
	month = mar,
	year = {2023},
	note = {arXiv:2303.11606 [cs]},
	keywords = {\#nosource, Computer Science - Computer Vision and Pattern Recognition},
}

Downloads: 0

{"_id":"A9tn5TPadBqGS3i6E","bibbaseid":"ju-noh-wang-seo-choi-cafsclassadaptiveframeworkforsemisupervisedsemanticsegmentation-2023","author_short":["Ju, J.","Noh, H.","Wang, Y.","Seo, M.","Choi, D."],"bibdata":{"bibtype":"misc","type":"misc","title":"CAFS: class adaptive framework for semi-supervised semantic segmentation","shorttitle":"Cafs","url":"http://arxiv.org/abs/2303.11606","abstract":"Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by selftraining with pseudo-labeling pixels having high confidences for unlabeled images. However, using only highconfidence pixels for self-training may result in losing much of the information in the unlabeled datasets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) to cope with the loss of most information that occurs in existing high-confidence-based pseudolabeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class. On this basis, we propose a calibration aware class-wise adaptive thresholding and classwise adaptive oversampling using the analysis results from the validation set. Our proposed CAFS achieves state-ofthe-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0% and 80.4%, respectively. The code is available at https://github.com/cjf8899/CAFS.","language":"en","urldate":"2023-05-10","publisher":"arXiv","author":[{"propositions":[],"lastnames":["Ju"],"firstnames":["Jingi"],"suffixes":[]},{"propositions":[],"lastnames":["Noh"],"firstnames":["Hyeoncheol"],"suffixes":[]},{"propositions":[],"lastnames":["Wang"],"firstnames":["Yooseung"],"suffixes":[]},{"propositions":[],"lastnames":["Seo"],"firstnames":["Minseok"],"suffixes":[]},{"propositions":[],"lastnames":["Choi"],"firstnames":["Dong-Geol"],"suffixes":[]}],"month":"March","year":"2023","note":"arXiv:2303.11606 [cs]","keywords":"#nosource, Computer Science - Computer Vision and Pattern Recognition","bibtex":"@misc{ju_cafs_2023,\n\ttitle = {{CAFS}: class adaptive framework for semi-supervised semantic segmentation},\n\tshorttitle = {Cafs},\n\turl = {http://arxiv.org/abs/2303.11606},\n\tabstract = {Semi-supervised semantic segmentation learns a model for classifying pixels into specific classes using a few labeled samples and numerous unlabeled images. The recent leading approach is consistency regularization by selftraining with pseudo-labeling pixels having high confidences for unlabeled images. However, using only highconfidence pixels for self-training may result in losing much of the information in the unlabeled datasets due to poor confidence calibration of modern deep learning networks. In this paper, we propose a class-adaptive semisupervision framework for semi-supervised semantic segmentation (CAFS) to cope with the loss of most information that occurs in existing high-confidence-based pseudolabeling methods. Unlike existing semi-supervised semantic segmentation frameworks, CAFS constructs a validation set on a labeled dataset, to leverage the calibration performance for each class. On this basis, we propose a calibration aware class-wise adaptive thresholding and classwise adaptive oversampling using the analysis results from the validation set. Our proposed CAFS achieves state-ofthe-art performance on the full data partition of the base PASCAL VOC 2012 dataset and on the 1/4 data partition of the Cityscapes dataset with significant margins of 83.0\\% and 80.4\\%, respectively. The code is available at https://github.com/cjf8899/CAFS.},\n\tlanguage = {en},\n\turldate = {2023-05-10},\n\tpublisher = {arXiv},\n\tauthor = {Ju, Jingi and Noh, Hyeoncheol and Wang, Yooseung and Seo, Minseok and Choi, Dong-Geol},\n\tmonth = mar,\n\tyear = {2023},\n\tnote = {arXiv:2303.11606 [cs]},\n\tkeywords = {\\#nosource, Computer Science - Computer Vision and Pattern Recognition},\n}\n\n\n\n","author_short":["Ju, J.","Noh, H.","Wang, Y.","Seo, M.","Choi, D."],"key":"ju_cafs_2023","id":"ju_cafs_2023","bibbaseid":"ju-noh-wang-seo-choi-cafsclassadaptiveframeworkforsemisupervisedsemanticsegmentation-2023","role":"author","urls":{"Paper":"http://arxiv.org/abs/2303.11606"},"keyword":["#nosource","Computer Science - Computer Vision and Pattern Recognition"],"metadata":{"authorlinks":{}},"html":""},"bibtype":"misc","biburl":"https://bibbase.org/zotero/fsimonetta","dataSources":["pzyFFGWvxG2bs63zP"],"keywords":["#nosource","computer science - computer vision and pattern recognition"],"search_terms":["cafs","class","adaptive","framework","semi","supervised","semantic","segmentation","ju","noh","wang","seo","choi"],"title":"CAFS: class adaptive framework for semi-supervised semantic segmentation","year":2023}