# Meta-hallucinator: Towards Few-Shot Cross-Modality Cardiac Image Segmentation

arXiv: https://arxiv.org/abs/2305.06978 · HTML: https://mlsanity.com/paper/2305.06978/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 11 May 2023.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Transformation Consistent Self-ensembling Model for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/1903.00348/) (2019)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [Unsupervised Bidirectional Cross-Modality Adaptation via Deeply Synergistic Image and Feature Alignment for Medical Image Segmentation](https://mlsanity.com/paper/2002.02255/) (2020)
- [Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation](https://mlsanity.com/paper/2007.09886/) (2020)
- [Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains](https://mlsanity.com/paper/2007.02035/) (2020)
- [SESS: Self-Ensembling Semi-Supervised 3D Object Detection](https://mlsanity.com/paper/1912.11803/) (2019)
- [Recurrent Mask Refinement for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2108.00622/) (2021)
- [Semi-supervised Contrastive Learning for Label-efficient Medical Image Segmentation](https://mlsanity.com/paper/2109.07407/) (2021)
- [Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation](https://mlsanity.com/paper/1907.13590/) (2019)
- [Image-and-Spatial Transformer Networks for Structure-Guided Image Registration](https://mlsanity.com/paper/1907.09200/) (2019)
- [DSAL: Deeply Supervised Active Learning from Strong and Weak Labelers for Biomedical Image Segmentation](https://mlsanity.com/paper/2101.09057/) (2021)
- [MT-UDA: Towards Unsupervised Cross-modality Medical Image Segmentation with Limited Source Labels](https://mlsanity.com/paper/2203.12454/) (2022)
- [Hierarchical Consistency Regularized Mean Teacher for Semi-supervised 3D Left Atrium Segmentation](https://mlsanity.com/paper/2105.10369/) (2021)
- [Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis](https://mlsanity.com/paper/2106.12499/) (2021)
- [Domain Generalizer: A Few-shot Meta Learning Framework for Domain Generalization in Medical Imaging](https://mlsanity.com/paper/2008.07724/) (2020)
- [Unsupervised Domain Adaptation for Small Bowel Segmentation using Disentangled Representation](https://mlsanity.com/paper/2107.02854/) (2021)
- [Image-level Harmonization of Multi-Site Data using Image-and-Spatial Transformer Networks](https://mlsanity.com/paper/2006.16741/) (2020)

## Built on by (most cited first, 2 of 2)

- [A Systematic Review of Few-Shot Learning in Medical Imaging](https://mlsanity.com/paper/2309.11433/) (2023)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)

## Abstract (arXiv)

Domain shift and label scarcity heavily limit deep learning applications to various medical image analysis tasks. Unsupervised domain adaptation (UDA) techniques have recently achieved promising cross-modality medical image segmentation by transferring knowledge from a label-rich source domain to an unlabeled target domain. However, it is also difficult to collect annotations from the source domain in many clinical applications, rendering most prior works suboptimal with the label-scarce source domain, particularly for few-shot scenarios, where only a few source labels are accessible. To achieve efficient few-shot cross-modality segmentation, we propose a novel transformation-consistent meta-hallucination framework, meta-hallucinator, with the goal of learning to diversify data distributions and generate useful examples for enhancing cross-modality performance. In our framework, hallucination and segmentation models are jointly trained with the gradient-based meta-learning strategy to synthesize examples that lead to good segmentation performance on the target domain. To further facilitate data hallucination and cross-domain knowledge transfer, we develop a self-ensembling model with a hallucination-consistent property. Our meta-hallucinator can seamlessly collaborate with the meta-segmenter for learning to hallucinate with mutual benefits from a combined view of meta-learning and self-ensembling learning. Extensive studies on MM-WHS 2017 dataset for cross-modality cardiac segmentation demonstrate that our method performs favorably against various approaches by a lot in the few-shot UDA scenario.