BrainVista: Modeling Naturalistic Brain Dynamics as Multimodal Next-Token Prediction
Not on the map. Why?
Abstract (arXiv)
Naturalistic fMRI characterizes the brain as a dynamic predictive engine driven by continuous sensory streams. However, modeling the causal forward evolution in realistic neural simulation is impeded by the timescale mismatch between multimodal inputs and the complex topology of cortical networks. To address these challenges, we introduce BrainVista, a multimodal autoregressive framework designed to model the causal evolution of brain states. BrainVista incorporates Network-wise Tokenizers to disentangle system-specific dynamics and a Spatial Mixer Head that captures inter-network information flow without compromising functional boundaries. Furthermore, we propose a novel Stimulus-to-Brain (S2B) masking mechanism to synchronize high-frequency sensory stimuli with hemodynamically filtered signals, enabling strict, history-only causal conditioning. We validate our framework on Algonauts 2025, CineBrain, and HAD, achieving state-of-the-art fMRI encoding performance. In long-horizon rollout settings, our model yields substantial improvements over baselines, increasing pattern correlation by 36.0% and 33.3% on relative to the strongest baseline Algonauts 2025 and CineBrain, respectively.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 53 references that are in the map. First posted on arXiv 4 Feb 2026.
Builds on
22 of 53 references.
References · 22
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 1,981
- 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,030
- 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 906
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
- 2023Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorscited by 156
- 2024MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Datacited by 105
- 2024Timer-XL: Long-Context Transformers for Unified Time Series Forecastingcited by 83
- 2024The Algonauts Project 2025 Challenge: How the Human Brain Makes Sense of Multimodal Moviescited by 19
- 2025TRIBE: TRImodal Brain Encoder for whole-brain fMRI response predictioncited by 17
10 more
- 2025MindSimulator: Exploring Brain Concept Localization via Synthetic FMRIcited by 15
- 2025Brain Harmony: A Multimodal Foundation Model Unifying Morphology and Function into 1D Tokenscited by 15
- 2025MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decodingcited by 14
- 2025CineBrain: A Large-Scale Multi-Modal Brain Dataset During Naturalistic Audiovisual Narrative Processingcited by 11
- 2025SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learningcited by 9
- 2025SLIM-Brain: A Data- and Training-Efficient Foundation Model for fMRI Data Analysiscited by 9
- 2024Uncovering cognitive taskonomy through transfer learning in masked autoencoder-based fMRI reconstructioncited by 7
- 2025Hierarchical Characterization of Brain Dynamics via State Space-based Vector Quantizationcited by 2
- 2025Improving Multimodal Brain Encoding Model with Dynamic Subject-awareness Routingcited by 2
- 2024BrainCodec: Neural fMRI codec for the decoding of cognitive brain statescited by 1
Built on by
Not cited yet.