# Large-scale modality-invariant foundation models for brain MRI analysis: Application to lesion segmentation

arXiv: https://arxiv.org/abs/2511.11311 · HTML: https://mlsanity.com/paper/2511.11311/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 14 Nov 2025.

## Builds on (most cited first, 10 of 10)

- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Automated brain extraction of multi-sequence MRI using artificial neural networks](https://mlsanity.com/paper/1901.11341/) (2019)
- [ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset](https://mlsanity.com/paper/2206.06694/) (2022)
- [An OpenMind for 3D medical vision self-supervised learning](https://mlsanity.com/paper/2412.17041/) (2024)
- [A large-scale heterogeneous 3D magnetic resonance brain imaging dataset for self-supervised learning](https://mlsanity.com/paper/2506.14432/) (2025)
- [Towards a Universal 3D Medical Multi-modality Generalization via Learning Personalized Invariant Representation](https://mlsanity.com/paper/2411.06106/) (2024)
- [Modality-Agnostic Learning for Medical Image Segmentation Using Multi-modality Self-distillation](https://mlsanity.com/paper/2306.03730/) (2023)
- [Unified 3D MRI Representations via Sequence-Invariant Contrastive Learning](https://mlsanity.com/paper/2501.12057/) (2025)

## Abstract (arXiv)

The field of computer vision is undergoing a paradigm shift toward large-scale foundation model pre-training via self-supervised learning (SSL). Leveraging large volumes of unlabeled brain MRI data, such models can learn anatomical priors that improve few-shot performance in diverse neuroimaging tasks. However, most SSL frameworks are tailored to natural images, and their adaptation to capture multi-modal MRI information remains underexplored. This work proposes a modality-invariant representation learning setup and evaluates its effectiveness in stroke and epilepsy lesion segmentation, following large-scale pre-training. Experimental results suggest that despite successful cross-modality alignment, lesion segmentation primarily benefits from preserving fine-grained modality-specific features. Model checkpoints and code are made publicly available.