# Anomaly Detection-Inspired Few-Shot Medical Image Segmentation Through Self-Supervision With Supervoxels

arXiv: https://arxiv.org/abs/2203.02048 · HTML: https://mlsanity.com/paper/2203.02048/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 68 references that are in the map.
- First posted on arXiv 3 Mar 2022.

## Builds on (most cited first, 30 of 39)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment](https://mlsanity.com/paper/1908.06391/) (2019)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [TADAM: Task dependent adaptive metric for improved few-shot learning](https://mlsanity.com/paper/1805.10123/) (2018)
- [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning](https://mlsanity.com/paper/1903.02351/) (2019)
- [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021)
- [SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation](https://mlsanity.com/paper/1810.09091/) (2018)
- [Part-aware Prototype Network for Few-shot Semantic Segmentation](https://mlsanity.com/paper/2007.06309/) (2020)
- [Multivariate mixture model for myocardium segmentation combining multi-source images](https://mlsanity.com/paper/1612.08820/) (2016)
- [Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation](https://mlsanity.com/paper/2007.09886/) (2020)

## Built on by (most cited first, 30 of 42)

- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [Few-Shot Medical Image Segmentation via a Region-enhanced Prototypical Transformer](https://mlsanity.com/paper/2309.04825/) (2023)
- [A Systematic Review of Few-Shot Learning in Medical Imaging](https://mlsanity.com/paper/2309.11433/) (2023)
- [Self-Supervised Anomaly Detection in Computer Vision and Beyond: A Survey and Outlook](https://mlsanity.com/paper/2205.05173/) (2022)
- [Q-Net: Query-Informed Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2208.11451/) (2022)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [Retrieval-augmented Few-shot Medical Image Segmentation with Foundation Models](https://mlsanity.com/paper/2408.08813/) (2024)
- [Few Shot Medical Image Segmentation with Cross Attention Transformer](https://mlsanity.com/paper/2303.13867/) (2023)
- [Meta-learning in healthcare: A survey](https://mlsanity.com/paper/2308.02877/) (2023)
- [Partition-A-Medical-Image: Extracting Multiple Representative Sub-regions for Few-shot Medical Image Segmentation](https://mlsanity.com/paper/2309.11172/) (2023)
- [SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting](https://mlsanity.com/paper/2411.17363/) (2024)
- [SynPo: Boosting Training-Free Few-Shot Medical Segmentation via High-Quality Negative Prompts](https://mlsanity.com/paper/2506.15153/) (2025)
- [Recent Advances in Medical Imaging Segmentation: A Survey](https://mlsanity.com/paper/2505.09274/) (2025)
- [RevSAM2: Prompt SAM2 for Medical Image Segmentation via Reverse-Propagation without Fine-tuning](https://mlsanity.com/paper/2409.04298/) (2024)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [RobustEMD: Domain Robust Matching for Cross-domain Few-shot Medical Image Segmentation](https://mlsanity.com/paper/2410.01110/) (2024)
- [A clinically motivated self-supervised approach for content-based image retrieval of CT liver images](https://mlsanity.com/paper/2207.04812/) (2022)
- [Rethinking Few-Shot Medical Image Segmentation by SAM2: A Training-Free Framework with Augmentative Prompting and Dynamic Matching](https://mlsanity.com/paper/2503.04826/) (2025)
- [Self-Supervised Few-Shot Learning for Ischemic Stroke Lesion Segmentation](https://mlsanity.com/paper/2303.01332/) (2023)
- [AutoPaint: A Self-Inpainting Method for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2305.12358/) (2023)
- [Concentrate on Weakness: Mining Hard Prototypes for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2505.21897/) (2025)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [Enhancing Weakly Supervised 3D Medical Image Segmentation through Probabilistic-aware Learning](https://mlsanity.com/paper/2403.02566/) (2024)
- [Few-Shot Medical Image Segmentation with Large Kernel Attention](https://mlsanity.com/paper/2407.19148/) (2024)
- [Tied Prototype Model for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2506.22101/) (2025)
- [Contrastive Graph Modeling for Cross-Domain Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2512.21683/) (2025)
- [Focus on Background: Exploring SAM's Potential in Few-shot Medical Image Segmentation with Background-centric Prompting](https://mlsanity.com/paper/2603.21287/) (2026)
- [DyABD: The Abdominal Muscle Segmentation in Dynamic MRI Benchmark](https://mlsanity.com/paper/2604.23187/) (2026)
- [Efficient Subclass Segmentation in Medical Images](https://mlsanity.com/paper/2307.00257/) (2023)
- [Support-Query Prototype Fusion Network for Few-shot Medical Image Segmentation](https://mlsanity.com/paper/2405.07516/) (2024)

## Abstract (arXiv)

Recent work has shown that label-efficient few-shot learning through self-supervision can achieve promising medical image segmentation results. However, few-shot segmentation models typically rely on prototype representations of the semantic classes, resulting in a loss of local information that can degrade performance. This is particularly problematic for the typically large and highly heterogeneous background class in medical image segmentation problems. Previous works have attempted to address this issue by learning additional prototypes for each class, but since the prototypes are based on a limited number of slices, we argue that this ad-hoc solution is insufficient to capture the background properties. Motivated by this, and the observation that the foreground class (e.g., one organ) is relatively homogeneous, we propose a novel anomaly detection-inspired approach to few-shot medical image segmentation in which we refrain from modeling the background explicitly. Instead, we rely solely on a single foreground prototype to compute anomaly scores for all query pixels. The segmentation is then performed by thresholding these anomaly scores using a learned threshold. Assisted by a novel self-supervision task that exploits the 3D structure of medical images through supervoxels, our proposed anomaly detection-inspired few-shot medical image segmentation model outperforms previous state-of-the-art approaches on two representative MRI datasets for the tasks of abdominal organ segmentation and cardiac segmentation.