# Prototype Mixture Models for Few-shot Semantic Segmentation

arXiv: https://arxiv.org/abs/2008.03898 · HTML: https://mlsanity.com/paper/2008.03898/

## Key facts

- Cited by 190 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Aug 2020.

## Built on by (most cited first, 30 of 190)

- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [Hypercorrelation Squeeze for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01538/) (2021)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Few-Shot Segmentation via Cycle-Consistent Transformer](https://mlsanity.com/paper/2106.02320/) (2021)
- [Learning What Not to Segment: A New Perspective on Few-Shot Segmentation](https://mlsanity.com/paper/2203.07615/) (2022)
- [Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer](https://mlsanity.com/paper/2108.03032/) (2021)
- [Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2207.10866/) (2022)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Self-Guided and Cross-Guided Learning for Few-Shot Segmentation](https://mlsanity.com/paper/2103.16129/) (2021)
- [Hierarchical Dense Correlation Distillation for Few-Shot Segmentation](https://mlsanity.com/paper/2303.14652/) (2023)
- [FS6D: Few-Shot 6D Pose Estimation of Novel Objects](https://mlsanity.com/paper/2203.14628/) (2022)
- [Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection](https://mlsanity.com/paper/2103.04612/) (2021)
- [VRP-SAM: SAM with Visual Reference Prompt](https://mlsanity.com/paper/2402.17726/) (2024)
- [Mining Latent Classes for Few-shot Segmentation](https://mlsanity.com/paper/2103.15402/) (2021)
- [Dense Cross-Query-and-Support Attention Weighted Mask Aggregation for Few-Shot Segmentation](https://mlsanity.com/paper/2207.08549/) (2022)
- [Feature-Proxy Transformer for Few-Shot Segmentation](https://mlsanity.com/paper/2210.06908/) (2022)
- [Learning Non-target Knowledge for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2205.04903/) (2022)
- [Learning Meta-class Memory for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2108.02958/) (2021)
- [Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuning](https://mlsanity.com/paper/2206.06122/) (2022)
- [Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation](https://mlsanity.com/paper/2106.11958/) (2021)
- [Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2204.10638/) (2022)
- [Open-vocabulary Semantic Segmentation with Frozen Vision-Language Models](https://mlsanity.com/paper/2210.15138/) (2022)
- [Intermediate Prototype Mining Transformer for Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2210.06780/) (2022)
- [Few-shot Segmentation with Optimal Transport Matching and Message Flow](https://mlsanity.com/paper/2108.08518/) (2021)
- [PFENet++: Boosting Few-shot Semantic Segmentation with the Noise-filtered Context-aware Prior Mask](https://mlsanity.com/paper/2109.13788/) (2021)
- [Unsupervised Video Object Segmentation via Prototype Memory Network](https://mlsanity.com/paper/2209.03712/) (2022)
- [Cross-Domain Few-Shot Segmentation via Iterative Support-Query Correspondence Mining](https://mlsanity.com/paper/2401.08407/) (2024)
- [Revealing Key Details to See Differences: A Novel Prototypical Perspective for Skeleton-based Action Recognition](https://mlsanity.com/paper/2411.18941/) (2024)

## Abstract (arXiv)

Few-shot segmentation is challenging because objects within the support and query images could significantly differ in appearance and pose. Using a single prototype acquired directly from the support image to segment the query image causes semantic ambiguity. In this paper, we propose prototype mixture models (PMMs), which correlate diverse image regions with multiple prototypes to enforce the prototype-based semantic representation. Estimated by an Expectation-Maximization algorithm, PMMs incorporate rich channel-wised and spatial semantics from limited support images. Utilized as representations as well as classifiers, PMMs fully leverage the semantics to activate objects in the query image while depressing background regions in a duplex manner. Extensive experiments on Pascal VOC and MS-COCO datasets show that PMMs significantly improve upon state-of-the-arts. Particularly, PMMs improve 5-shot segmentation performance on MS-COCO by up to 5.82% with only a moderate cost for model size and inference speed.