# Masked and Predictive Self-Supervised Foundation Models for 3D Brain MRI

arXiv: https://arxiv.org/abs/2606.13315 · HTML: https://mlsanity.com/paper/2606.13315/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 67 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 18 of 18)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [Revisiting Feature Prediction for Learning Visual Representations from Video](https://mlsanity.com/paper/2404.08471/) (2024)
- [Better Fine-Tuning by Reducing Representational Collapse](https://mlsanity.com/paper/2008.03156/) (2020)
- [Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision](https://mlsanity.com/paper/2310.18689/) (2023)
- [The 2024 Brain Tumor Segmentation (BraTS) Challenge: Glioma Segmentation on Post-treatment MRI](https://mlsanity.com/paper/2405.18368/) (2024)
- [Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking](https://mlsanity.com/paper/2409.19407/) (2024)
- [A Survey on Trustworthiness in Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2407.15851/) (2024)
- [Video Representation Learning with Joint-Embedding Predictive Architectures](https://mlsanity.com/paper/2412.10925/) (2024)
- [MRI-CORE: A Foundation Model for Magnetic Resonance Imaging](https://mlsanity.com/paper/2506.12186/) (2025)
- [Medical Slice Transformer: Improved Diagnosis and Explainability on 3D Medical Images with DINOv2](https://mlsanity.com/paper/2411.15802/) (2024)
- [Exploring Representation Invariance in Finetuning](https://mlsanity.com/paper/2503.07399/) (2025)
- [US-JEPA: A Joint Embedding Predictive Architecture for Ultrasound](https://mlsanity.com/paper/2602.19322/) (2026)
- [Towards Fair Medical AI: Adversarial Debiasing of 3D CT Foundation Embeddings](https://mlsanity.com/paper/2502.04386/) (2025)
- [Self-Supervised Learning for Knee Osteoarthritis: Diagnostic Limitations and Prognostic Value of Hospital Data](https://mlsanity.com/paper/2603.24903/) (2026)

## Built on by (most cited first, 1 of 1)

- [Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography](https://mlsanity.com/paper/2607.26196/) (2026)

## Abstract (arXiv)

Self-supervised foundation models have shown strong promise in medical imaging. However, existing MRI foundation-model studies have primarily emphasized segmentation and dense prediction tasks, while systematic investigation of self-supervised foundation models for MRI-based disease detection remains limited. In this work, we investigate two major self-supervised pretraining paradigms for MRI-based disease detection: reconstruction-based learning via Masked Autoencoders (MAE) and predictive representation learning via Joint Embedding Predictive Architectures (JEPA). We study the role of auxiliary objectives by introducing a novel spectral-domain reconstruction loss for MAE to enhance sensitivity to fine-grained anatomical structure, and by integrating variance--covariance regularization (VCR) within our JEPA framework to encourage decorrelated latent representations. Our models are pretrained on heterogeneous single-contrast MRI volumes in a contrast-agnostic setting, without modality concatenation. Across five downstream disease detection tasks, our results highlight the importance of self-supervised objective design for medical foundation model pretraining, demonstrating that the downstream benefit of each objective is determined by its relevance to the task's structure. Specifically, spectral regularization yields the largest improvements when the downstream discriminative signal is characterized by strong high-frequency anatomical structures, while covariance regularization is most beneficial when discriminative information spans multiple decorrelated feature dimensions. MAE with spectral-domain supervision consistently achieves superior downstream performance for MRI-based disease detection. These findings suggest that self-supervised objectives in medical imaging encode specific biases, and their downstream benefit is fundamentally conditioned on the task's structure.