Learning Meta-class Memory for Few-Shot Semantic Segmentation

Learning Meta-class Memory for Few-Shot Semantic Segmentation. Wu, Z., Shi, X., lin, G., & Cai, J. 2021.

Learning Meta-class Memory for Few-Shot Semantic Segmentation [link]

Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\% higher than the previous state-of-the-art.

@article{
 title = {Learning Meta-class Memory for Few-Shot Semantic Segmentation},
 type = {article},
 year = {2021},
 pages = {517-526},
 websites = {http://arxiv.org/abs/2108.02958},
 id = {d5e3e8d3-4676-3568-8646-52a9df7fa357},
 created = {2021-10-30T07:28:03.974Z},
 file_attached = {true},
 profile_id = {f3d36c73-062b-3738-9a74-d09e4e83eb1e},
 group_id = {1ff583c0-be37-34fa-9c04-73c69437d354},
 last_modified = {2021-10-30T07:28:39.005Z},
 read = {false},
 starred = {false},
 authored = {false},
 confirmed = {true},
 hidden = {false},
 folder_uuids = {48c2017d-ceec-4fcb-b966-b19e6f311352},
 private_publication = {false},
 abstract = {Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\% higher than the previous state-of-the-art.},
 bibtype = {article},
 author = {Wu, Zhonghua and Shi, Xiangxi and lin, Guosheng and Cai, Jianfei}
}

Downloads: 0

{"_id":"pRHH9siS4JrxbSvGr","bibbaseid":"wu-shi-lin-cai-learningmetaclassmemoryforfewshotsemanticsegmentation-2021","author_short":["Wu, Z.","Shi, X.","lin, G.","Cai, J."],"bibdata":{"title":"Learning Meta-class Memory for Few-Shot Semantic Segmentation","type":"article","year":"2021","pages":"517-526","websites":"http://arxiv.org/abs/2108.02958","id":"d5e3e8d3-4676-3568-8646-52a9df7fa357","created":"2021-10-30T07:28:03.974Z","file_attached":"true","profile_id":"f3d36c73-062b-3738-9a74-d09e4e83eb1e","group_id":"1ff583c0-be37-34fa-9c04-73c69437d354","last_modified":"2021-10-30T07:28:39.005Z","read":false,"starred":false,"authored":false,"confirmed":"true","hidden":false,"folder_uuids":"48c2017d-ceec-4fcb-b966-b19e6f311352","private_publication":false,"abstract":"Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\\% higher than the previous state-of-the-art.","bibtype":"article","author":"Wu, Zhonghua and Shi, Xiangxi and lin, Guosheng and Cai, Jianfei","bibtex":"@article{\n title = {Learning Meta-class Memory for Few-Shot Semantic Segmentation},\n type = {article},\n year = {2021},\n pages = {517-526},\n websites = {http://arxiv.org/abs/2108.02958},\n id = {d5e3e8d3-4676-3568-8646-52a9df7fa357},\n created = {2021-10-30T07:28:03.974Z},\n file_attached = {true},\n profile_id = {f3d36c73-062b-3738-9a74-d09e4e83eb1e},\n group_id = {1ff583c0-be37-34fa-9c04-73c69437d354},\n last_modified = {2021-10-30T07:28:39.005Z},\n read = {false},\n starred = {false},\n authored = {false},\n confirmed = {true},\n hidden = {false},\n folder_uuids = {48c2017d-ceec-4fcb-b966-b19e6f311352},\n private_publication = {false},\n abstract = {Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the $k$-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-$5^i$ and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5\\% mIoU on the COCO dataset in 1-shot setting, which is 5.1\\% higher than the previous state-of-the-art.},\n bibtype = {article},\n author = {Wu, Zhonghua and Shi, Xiangxi and lin, Guosheng and Cai, Jianfei}\n}","author_short":["Wu, Z.","Shi, X.","lin, G.","Cai, J."],"urls":{"Paper":"https://bibbase.org/service/mendeley/bfbbf840-4c42-3914-a463-19024f50b30c/file/4d097388-ca08-49ff-2db5-d30811eeb266/Wu_Learning_Meta_Class_Memory_for_Few_Shot_Semantic_Segmentation_ICCV_2021_paper.pdf.pdf","Website":"http://arxiv.org/abs/2108.02958"},"biburl":"https://bibbase.org/service/mendeley/bfbbf840-4c42-3914-a463-19024f50b30c","bibbaseid":"wu-shi-lin-cai-learningmetaclassmemoryforfewshotsemanticsegmentation-2021","role":"author","metadata":{"authorlinks":{}},"downloads":0},"bibtype":"article","biburl":"https://bibbase.org/service/mendeley/bfbbf840-4c42-3914-a463-19024f50b30c","dataSources":["2252seNhipfTmjEBQ"],"keywords":[],"search_terms":["learning","meta","class","memory","few","shot","semantic","segmentation","wu","shi","lin","cai"],"title":"Learning Meta-class Memory for Few-Shot Semantic Segmentation","year":2021}