# Enhancing Modality-Agnostic Representations via Meta-Learning for Brain Tumor Segmentation

arXiv: https://arxiv.org/abs/2302.04308 · HTML: https://mlsanity.com/paper/2302.04308/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 90 references that are in the map.
- First posted on arXiv 8 Feb 2023.

## Builds on (most cited first, 30 of 49)

- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Learning to Generalize: Meta-Learning for Domain Generalization](https://mlsanity.com/paper/1710.03463/) (2017)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [HeMIS: Hetero-Modal Image Segmentation](https://mlsanity.com/paper/1607.05194/) (2016)
- [Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network](https://mlsanity.com/paper/1904.12200/) (2019)
- [mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentation](https://mlsanity.com/paper/2206.02425/) (2022)
- [Annotation-efficient deep learning for automatic medical image segmentation](https://mlsanity.com/paper/2012.04885/) (2020)
- [Topology-Aware Segmentation Using Discrete Morse Theory](https://mlsanity.com/paper/2103.09992/) (2021)
- [Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentation](https://mlsanity.com/paper/1907.11150/) (2019)
- [Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion](https://mlsanity.com/paper/2002.09708/) (2020)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2106.14591/) (2021)
- [Brain Tumor Segmentation on MRI with Missing Modalities](https://mlsanity.com/paper/1904.07290/) (2019)
- [Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalities](https://mlsanity.com/paper/2104.06231/) (2021)
- [Trigger Hunting with a Topological Prior for Trojan Detection](https://mlsanity.com/paper/2110.08335/) (2021)
- [SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalities](https://mlsanity.com/paper/2204.02961/) (2022)

## Built on by (most cited first, 14 of 14)

- [Multi-modal Vision Pre-training for Medical Image Analysis](https://mlsanity.com/paper/2410.10604/) (2024)
- [Enhancing SAM with Efficient Prompting and Preference Optimization for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2503.04639/) (2025)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Unified-modal Salient Object Detection via Adaptive Prompt Learning](https://mlsanity.com/paper/2311.16835/) (2023)
- [ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalities](https://mlsanity.com/paper/2407.03836/) (2024)
- [Meta-Learned Modality-Weighted Knowledge Distillation for Robust Multi-Modal Learning with Missing Data](https://mlsanity.com/paper/2405.07155/) (2024)
- [Spatial Diffusion for Cell Layout Generation](https://mlsanity.com/paper/2409.03106/) (2024)
- [Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization](https://mlsanity.com/paper/2406.01987/) (2024)
- [Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2408.13733/) (2024)
- [SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement](https://mlsanity.com/paper/2412.06352/) (2024)
- [Learning Topological Representations for Deep Image Understanding](https://mlsanity.com/paper/2403.15361/) (2024)
- [Robust Brain Tumor Segmentation with Incomplete MRI Modalities Using H"older Divergence and Mutual Information-Enhanced Knowledge Transfer](https://mlsanity.com/paper/2507.01254/) (2025)
- [brat: Aligned Multi-View Embeddings for Brain MRI Analysis](https://mlsanity.com/paper/2512.18679/) (2025)
- [D3Seg: Dependency-Aware Diffusion for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2605.22249/) (2026)

## Abstract (arXiv)

In medical vision, different imaging modalities provide complementary information. However, in practice, not all modalities may be available during inference or even training. Previous approaches, e.g., knowledge distillation or image synthesis, often assume the availability of full modalities for all patients during training; this is unrealistic and impractical due to the variability in data collection across sites. We propose a novel approach to learn enhanced modality-agnostic representations by employing a meta-learning strategy in training, even when only limited full modality samples are available. Meta-learning enhances partial modality representations to full modality representations by meta-training on partial modality data and meta-testing on limited full modality samples. Additionally, we co-supervise this feature enrichment by introducing an auxiliary adversarial learning branch. More specifically, a missing modality detector is used as a discriminator to mimic the full modality setting. Our segmentation framework significantly outperforms state-of-the-art brain tumor segmentation techniques in missing modality scenarios.