# Towards a general-purpose foundation model for fMRI analysis

arXiv: https://arxiv.org/abs/2506.11167 · HTML: https://mlsanity.com/paper/2506.11167/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 71 references that are in the map.
- First posted on arXiv 11 Jun 2025.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors](https://mlsanity.com/paper/2305.18274/) (2023)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [On the Challenges and Perspectives of Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2306.05705/) (2023)
- [MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data](https://mlsanity.com/paper/2403.11207/) (2024)
- [Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects](https://mlsanity.com/paper/2306.10125/) (2023)
- [Brain Network Transformer](https://mlsanity.com/paper/2210.06681/) (2022)
- [Mind Reader: Reconstructing complex images from brain activities](https://mlsanity.com/paper/2210.01769/) (2022)
- [MindBridge: A Cross-Subject Brain Decoding Framework](https://mlsanity.com/paper/2404.07850/) (2024)
- [Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking](https://mlsanity.com/paper/2409.19407/) (2024)
- [Interpretable Graph Neural Networks for Connectome-Based Brain Disorder Analysis](https://mlsanity.com/paper/2207.00813/) (2022)
- [On Data Scaling in Masked Image Modeling](https://mlsanity.com/paper/2206.04664/) (2022)
- [SwiFT: Swin 4D fMRI Transformer](https://mlsanity.com/paper/2307.05916/) (2023)
- [Community-Aware Transformer for Autism Prediction in fMRI Connectome](https://mlsanity.com/paper/2307.10181/) (2023)
- [Attend and Decode: 4D fMRI Task State Decoding Using Attention Models](https://mlsanity.com/paper/2004.05234/) (2020)
- [Self-Supervised Transformers for fMRI representation](https://mlsanity.com/paper/2112.05761/) (2021)
- [AnyStar: Domain randomized universal star-convex 3D instance segmentation](https://mlsanity.com/paper/2307.07044/) (2023)
- [BrainChat: Decoding Semantic Information from fMRI using Vision-language Pretrained Models](https://mlsanity.com/paper/2406.07584/) (2024)
- [Modality-Agnostic fMRI Decoding of Vision and Language](https://mlsanity.com/paper/2403.11771/) (2024)
- [Mind-to-Image: Projecting Visual Mental Imagination of the Brain from fMRI](https://mlsanity.com/paper/2404.05468/) (2024)
- [Brain Dialogue Interface (BDI): A User-Friendly fMRI Model for Interactive Brain Decoding](https://mlsanity.com/paper/2407.09509/) (2024)
- [Large-scale Graph Representation Learning of Dynamic Brain Connectome with Transformers](https://mlsanity.com/paper/2312.14939/) (2023)

## Built on by (most cited first, 15 of 15)

- [SLIM-Brain: A Data- and Training-Efficient Foundation Model for fMRI Data Analysis](https://mlsanity.com/paper/2512.21881/) (2025)
- [FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI](https://mlsanity.com/paper/2606.11500/) (2026)
- [Scaling Vision Transformers for Functional MRI with Flat Maps](https://mlsanity.com/paper/2510.13768/) (2025)
- [BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamics](https://mlsanity.com/paper/2606.17742/) (2026)
- [Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Model](https://mlsanity.com/paper/2512.11582/) (2025)
- [World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications](https://mlsanity.com/paper/2606.00133/) (2026)
- [From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?](https://mlsanity.com/paper/2509.17280/) (2025)
- [Region-Aware Reconstruction Strategy for Pre-training fMRI Foundation Model](https://mlsanity.com/paper/2511.00443/) (2025)
- [MAC: Masked Agent Collaboration Boosts Large Language Model Medical Decision-Making](https://mlsanity.com/paper/2507.21159/) (2025)
- [Expertise-aware Multi-LLM Recruitment and Collaboration for Medical Decision-Making](https://mlsanity.com/paper/2508.13754/) (2025)
- [Prediction and Causality of functional MRI and synthetic signal using a Zero-Shot Time-Series Foundation Model](https://mlsanity.com/paper/2509.12497/) (2025)
- [Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis](https://mlsanity.com/paper/2602.11536/) (2026)
- [One Brain, Omni Modalities: Towards Unified Non-Invasive Brain Decoding with Large Language Models](https://mlsanity.com/paper/2602.21522/) (2026)
- [PIMSM: Physics-Informed Multi-Scale Mamba for Stable Neural Representations under Distribution Shift](https://mlsanity.com/paper/2605.16351/) (2026)
- [Learning Robust and Task-Invariant Functional Representation from fMRI through Siamese Self-Supervised Learning](https://mlsanity.com/paper/2605.28990/) (2026)

## Abstract (arXiv)

Functional MRI (fMRI) is crucial for studying brain function and diagnosing neurological disorders. However, existing analysis methods suffer from reproducibility and transferability challenges due to complex preprocessing pipelines and task-specific model designs. In this work, we introduce NeuroSTORM (Neuroimaging Foundation Model with Spatial-Temporal Optimized Representation Modeling) that learns generalizable representations directly from 4D fMRI volumes and enables efficient transfer to diverse downstream applications. Specifically, NeuroSTORM is pre-trained on 28.65 million fMRI frames from over 50,000 subjects, spanning multiple centers and ages 5 to 100. It combines an efficient spatiotemporal modeling design and lightweight task adaptation to enable scalable pre-training and fast transfer to downstream applications. Here we show that NeuroSTORM consistently outperforms existing methods across five downstream tasks, including demographic prediction, phenotype prediction, disease diagnosis, re-identification, and state classification. On two multi-hospital clinical cohorts with 17 diagnoses, NeuroSTORM achieves the best diagnosis performance while remaining predictive of psychological and cognitive phenotypes. These results suggest that NeuroSTORM could become a standardized foundation model for reproducible and transferable fMRI analysis.