# Learning Meta-class Memory for Few-Shot Semantic Segmentation

arXiv: https://arxiv.org/abs/2108.02958 · HTML: https://mlsanity.com/paper/2108.02958/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 6 Aug 2021.

## Builds on (most cited first, 28 of 28)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Prototype Mixture Models for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2008.03898/) (2020)
- [Feature Weighting and Boosting for Few-Shot Segmentation](https://mlsanity.com/paper/1909.13140/) (2019)
- [Part-aware Prototype Network for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2007.06309/) (2020)
- [Simultaneous Detection and Segmentation](https://mlsanity.com/paper/1407.1808/) (2014)
- [Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation](https://mlsanity.com/paper/2007.09886/) (2020)
- [CRNet: Cross-Reference Networks for Few-Shot Segmentation](https://mlsanity.com/paper/2003.10658/) (2020)
- [FGN: Fully Guided Network for Few-Shot Instance Segmentation](https://mlsanity.com/paper/2003.13954/) (2020)
- [M2E-Try On Net: Fashion from Model to Everyone](https://mlsanity.com/paper/1811.08599/) (2018)
- [SimPropNet: Improved Similarity Propagation for Few-shot Image Segmentation](https://mlsanity.com/paper/2004.15014/) (2020)
- [Exploring Bottom-up and Top-down Cues with Attentive Learning for Webly Supervised Object Detection](https://mlsanity.com/paper/2003.09790/) (2020)
- [Keypoint Based Weakly Supervised Human Parsing](https://mlsanity.com/paper/1809.05285/) (2018)
- [Remember What You have drawn: Semantic Image Manipulation with Memory](https://mlsanity.com/paper/2107.12579/) (2021)

## Built on by (most cited first, 30 of 56)

- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Exploring Smoothness and Class-Separation for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2203.01324/) (2022)
- [Feature-Proxy Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2210.06908/) (2022)
- [Learning Non-target Knowledge for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2205.04903/) (2022)
- [Pin the Memory: Learning to Generalize Semantic Segmentation](https://mlsanity.com/paper/2204.03609/) (2022)
- [Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2204.10638/) (2022)
- [Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2210.06780/) (2022)
- [PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask](https://mlsanity.com/paper/2109.13788/) (2021)
- [Beyond the Prototype: Divide-and-conquer Proxies for Few-shot Segmentation](https://mlsanity.com/paper/2204.09903/) (2022)
- [APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2406.08372/) (2024)
- [Integrative Few-Shot Learning for Classification and Segmentation](https://mlsanity.com/paper/2203.15712/) (2022)
- [MIANet: Aggregating Unbiased Instance and General Information for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2305.13864/) (2023)
- [Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation](https://mlsanity.com/paper/2307.03407/) (2023)
- [Mask Matching Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2301.01208/) (2022)
- [FECANet: Boosting Few-Shot Semantic Segmentation with Feature-Enhanced Context-Aware Network](https://mlsanity.com/paper/2301.08160/) (2023)
- [MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation](https://mlsanity.com/paper/2206.09667/) (2022)
- [Learning Expressive Prompting With Residuals for Vision Transformers](https://mlsanity.com/paper/2303.15591/) (2023)
- [Domain-General Crowd Counting in Unseen Scenarios](https://mlsanity.com/paper/2212.02573/) (2022)
- [Visual and Textual Prior Guided Mask Assemble for Few-Shot Segmentation and Beyond](https://mlsanity.com/paper/2308.07539/) (2023)
- [Dense Gaussian Processes for Few-Shot Segmentation](https://mlsanity.com/paper/2110.03674/) (2021)
- [Cost Aggregation Is All You Need for Few-Shot Segmentation](https://mlsanity.com/paper/2112.11685/) (2021)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation](https://mlsanity.com/paper/2207.09084/) (2022)
- [Cross-Image Region Mining with Region Prototypical Network for Weakly Supervised Segmentation](https://mlsanity.com/paper/2108.07413/) (2021)
- [A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models](https://mlsanity.com/paper/2401.11311/) (2024)
- [MFNet: Multi-class Few-shot Segmentation Network with Pixel-wise Metric Learning](https://mlsanity.com/paper/2111.00232/) (2021)
- [Iterative Few-shot Semantic Segmentation from Image Label Text](https://mlsanity.com/paper/2303.05646/) (2023)
- [Harmonizing Base and Novel Classes: A Class-Contrastive Approach for Generalized Few-Shot Segmentation](https://mlsanity.com/paper/2303.13724/) (2023)
- [A Self-Distillation Embedded Supervised Affinity Attention Model for Few-Shot Segmentation](https://mlsanity.com/paper/2108.06600/) (2021)

## Abstract (arXiv)

Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the k-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-5i and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5% mIoU on the COCO dataset in 1-shot setting, which is 5.1% higher than the previous state-of-the-art.