# Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretraining

arXiv: https://arxiv.org/abs/2604.12683 · HTML: https://mlsanity.com/paper/2604.12683/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 8 of its 33 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 8 of 8)

- [Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking](https://mlsanity.com/paper/2409.19407/) (2024)
- [Do text-free diffusion models learn discriminative visual representations?](https://mlsanity.com/paper/2311.17921/) (2023)
- [BrainMass: Advancing Brain Network Analysis for Diagnosis with Large-scale Self-Supervised Learning](https://mlsanity.com/paper/2403.01433/) (2024)
- [The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Function](https://mlsanity.com/paper/1511.02976/) (2015)
- [CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processing](https://mlsanity.com/paper/2503.06940/) (2025)
- [Omni-fMRI: A Universal Atlas-Free fMRI Foundation Model](https://mlsanity.com/paper/2601.23090/) (2026)
- [SLIM-Brain: A Data- and Training-Efficient Foundation Model for fMRI Data Analysis](https://mlsanity.com/paper/2512.21881/) (2025)
- [Uncovering cognitive taskonomy through transfer learning in masked autoencoder-based fMRI reconstruction](https://mlsanity.com/paper/2407.00033/) (2024)

## Built on by (most cited first, 11 of 11)

- [FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI](https://mlsanity.com/paper/2606.11500/) (2026)
- [BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics](https://mlsanity.com/paper/2606.17742/) (2026)
- [Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptation](https://mlsanity.com/paper/2605.07302/) (2026)
- [Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics](https://mlsanity.com/paper/2606.11833/) (2026)
- [Beyond Single-Source Cognitive Taskonomy:Multi-Source Task Relations through fMRI Transfer Learning](https://mlsanity.com/paper/2606.26279/) (2026)
- [BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models](https://mlsanity.com/paper/2609.10518/) (2026)
- [Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding](https://mlsanity.com/paper/2609.11341/) (2026)
- [Omni-Sleep: A Sleep Foundation Model via Hierarchical Contrastive Learning of CNS-ANS Dynamics](https://mlsanity.com/paper/2607.07720/) (2026)
- [A Scaling Study for fMRI Foundation Models](https://mlsanity.com/paper/2609.27232/) (2026)
- [FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps](https://mlsanity.com/paper/2609.31204/) (2026)
- [Natural Image Autoencoder-Based fMRI Representations for Trait and State Prediction](https://mlsanity.com/paper/2609.34167/) (2026)

## Abstract (arXiv)

Current fMRI foundation models primarily rely on a limited range of brain states and mismatched pretraining tasks, restricting their ability to learn generalized representations across diverse brain states. We present Brain-DiT, a universal multi-state fMRI foundation model pretrained on 349,898 sessions from 24 datasets spanning resting, task, naturalistic, disease, and sleep states. Unlike prior fMRI foundation models that rely on masked reconstruction in the raw-signal space or a latent space, Brain-DiT adopts metadata-conditioned diffusion pretraining with a Diffusion Transformer (DiT), enabling the model to learn multi-scale representations that capture both fine-grained functional structure and global semantics. Across extensive evaluations and ablations on 7 downstream tasks, we find consistent evidence that diffusion-based generative pretraining is a stronger proxy than reconstruction or alignment, with metadata-conditioned pretraining further improving downstream performance by disentangling intrinsic neural dynamics from population-level variability. We also observe that downstream tasks exhibit distinct preferences for representational scale: ADNI classification benefits more from global semantic representations, whereas age/sex prediction comparatively relies more on fine-grained local structure.