mlsanityData to 14 Oct 2026

A Modality-agnostic Multi-task Foundation Model for Human Brain Imaging

arXiv 2509.00549 ↗First posted 30 Aug 2025cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent learning-based approaches have made astonishing advances in calibrated medical imaging like computerized tomography (CT), yet they struggle to generalize in uncalibrated modalities -- notably magnetic resonance (MR) imaging, where performance is highly sensitive to the differences in MR contrast, resolution, and orientation. This prevents broad applicability to diverse real-world clinical protocols. Here we introduce BrainFM, a modality-agnostic, multi-task vision foundation model for human brain imaging. With the proposed "mild-to-severe" intra-subject generation and "real-synth" mix-up training strategy, BrainFM is resilient to the appearance of acquired images (e.g., modality, contrast, deformation, resolution, artifacts), and can be directly applied to five fundamental brain imaging tasks, including image synthesis for CT and T1w/T2w/FLAIR MRI, anatomy segmentation, scalp-to-cortical distance, bias field estimation, and registration. We evaluate the efficacy of BrainFM on eleven public datasets, and demonstrate its robustness and effectiveness across all tasks and input modalities. Code is available at https://github.com/jhuldr/BrainFM.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 32 of its 64 references that are in the map. First posted on arXiv 30 Aug 2025.

Builds on

32 of 64 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023Segment Anythingcited by 8,234
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  7. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  8. 2018VoxelMorph: A Learning Framework for Deformable Medical Image Registrationcited by 662
  9. 2018Deep Visual Domain Adaptation: A Surveycited by 635
  10. 2022MONAI: An open-source framework for deep learning in healthcarecited by 448
  11. 2023Towards Generalist Biomedical AIcited by 289
  12. 2016Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentationcited by 263
20 more
  1. 2021SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retrainingcited by 237
  2. 2018A Deep Learning Framework for Unsupervised Affine and Deformable Image Registrationcited by 160
  3. 2020SynthMorph: learning contrast-invariant registration without acquired imagescited by 140
  4. 2022SynthStrip: Skull-Stripping for Any Brain Imagecited by 127
  5. 2017Quicksilver: Fast Predictive Image Registration - a Deep Learning Approachcited by 113
  6. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
  7. 2020Local Temperature Scaling for Probability Calibrationcited by 95
  8. 2020SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Imagescited by 67
  9. 2021Momentum Contrastive Voxel-wise Representation Learning for Semi-supervised Volumetric Medical Image Segmentationcited by 49
  10. 2020Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrastcited by 45
  11. 2021Accurate Point Cloud Registration with Robust Optimal Transportcited by 38
  12. 2021FastSurferVINN: Building Resolution-Independence into Deep Learning Segmentation Methods -- A Solution for HighRes Brain MRIcited by 19
  13. 2020A Contrast-Adaptive Method for Simultaneous Whole-Brain and Lesion Segmentation in Multiple Sclerosiscited by 17
  14. 2023Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable MRIcited by 15
  15. 2023SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramidcited by 14
  16. 2020Discovering Hidden Physics Behind Transport Dynamicscited by 8
  17. 2024Registration by Regression (RbR): a framework for interpretable and flexible atlas registrationcited by 8
  18. 2024Recon-all-clinical: Cortical surface reconstruction and analysis of heterogeneous clinical brain MRIcited by 7
  19. 2024Hierarchical Uncertainty Estimation for Learning-based Registration in Neuroimagingcited by 5
  20. 2021Deep Decomposition for Stochastic Normal-Abnormal Transportcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Do 3D Medical Foundation Models See Through MRI Artifacts? A Controlled Study of Representation Robustnesscited by 1
  2. 2026Enhanced Portable Ultra Low-Field Diffusion Tensor Imaging with Bayesian Artifact Correction and Deep Learning-Based Super-Resolutioncited by 0
  3. 2026CAHAL: Clinically Applicable resolution enHAncement for Low-resolution MRI scanscited by 0
  4. 2026Cross-Modality Structural Guidance in 3D Latent Diffusion for Robust FLAIR Super-Resolutioncited by 0
  5. 2026ReMAP-PET: Beyond Visual Understanding -- Learning Region-Guided Metabolic Alignment Semantics from Brain PETcited by 0
  6. 2026Uncertainty-Aware 3D Residual Wavelet Diffusion for Ultra Low-Field MRI Super-Resolutioncited by 0
  7. 2026Dataset-Specific Encoding of Image Content Underlies Cross-Dataset Failure in Brain MRI Foundation Modelscited by 0
  8. 2026Quantifying privacy risks in foundation model representations of brain MRIcited by 0