mlsanityData to 7 Oct 2026

Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking

arXiv 2409.19407 ↗First posted 28 Sep 2024q-bio.NC cs.AI cs.CVcited by 64

Not on the map. Why?

Abstract (arXiv)

We introduce Brain-JEPA, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: Brain Gradient Positioning and Spatiotemporal Masking. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Builds on 20 of its 62 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

20 of 62 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  8. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  9. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  10. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  11. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  12. 2017Convolutional Sequence to Sequence Learningcited by 550
8 more
  1. 2022Brain Network Transformercited by 92
  2. 2022BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networkscited by 73
  3. 2022Self-Supervised Learning of Brain Dynamics from Broad Neuroimaging Datacited by 39
  4. 2023SwiFT: Swin 4D fMRI Transformercited by 36
  5. 2024BrainMass: Advancing Brain Network Analysis for Diagnosis with Large-scale Self-Supervised Learningcited by 32
  6. 2023Beyond the Snapshot: Brain Tokenized Graph Transformer for Longitudinal Brain Functional Connectome Embeddingcited by 6
  7. 2023SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixupcited by 3
  8. 2024Prompt Your Brain: Scaffold Prompt Tuning for Efficient Adaptation of fMRI Pre-trained Modelcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 22; 2026: 42.

64 papers.

Sort:

By area:

  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  2. 2025Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discoverycited by 22
  3. 2025Towards a general-purpose foundation model for fMRI analysiscited by 15
  4. 2026NeuralBench: A Unifying Framework to Benchmark NeuroAI Modelscited by 13
  5. 2026Brain-DiT: A Universal Multi-state fMRI Foundation Model with Metadata-Conditioned Pretrainingcited by 11
  6. 2026Subspace-Decomposed JEPAs: Disentangling Progression and Content in Latent World Modelscited by 11
  7. 20253D Foundation Model for Generalizable Disease Detection in Head Computed Tomographycited by 10
  8. 2025SLIM-Brain: A Data- and Training-Efficient Foundation Model for fMRI Data Analysiscited by 9
  9. 2025From Images to Signals: Are Large Vision Models Useful for Time Series Analysis?cited by 7
  10. 2025fMRI-LM: Towards a Universal Foundation Model for Language-Aligned fMRI Understandingcited by 7
  11. 2025BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activitycited by 7
  12. 2025ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Modelscited by 6
38 more
  1. 2026FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRIcited by 6
  2. 2025Voxel-Level Brain States Prediction Using Swin Transformercited by 5
  3. 2025Scaling Vision Transformers for Functional MRI with Flat Mapscited by 5
  4. 2026BrainWorld: A Structural-Prior-Conditioned Generative Model for Whole-Brain 4D fMRI Dynamicscited by 5
  5. 2025Large Connectome Model: An fMRI Foundation Model of Brain Connectomes Empowered by Brain-Environment Interaction in Multitask Learning Landscapecited by 4
  6. 2025CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Imagescited by 4
  7. 2025Brain-Semantoks: Learning Semantic Tokens of Brain Dynamics with a Self-Distilled Foundation Modelcited by 4
  8. 2025A Brain Graph Foundation Model: Pre-Training and Prompt-Tuning across Broad Atlases and Disorderscited by 3
  9. 2026Self-Distillation of Hidden Layers for Self-Supervised Representation Learningcited by 3
  10. 2026Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policiescited by 3
  11. 2025CytoNet: A Foundation Model for the Human Cerebral Cortex at Cellular Resolutioncited by 2
  12. 2026Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEGcited by 2
  13. 2026CGM-JEPA: Learning Consistent Continuous Glucose Monitor Representations via Predictive Self-Supervised Pretrainingcited by 2
  14. 2025Stochastic Optimal Control for Continuous-Time fMRI Representation Learningcited by 1
  15. 2025BrainStratify: Coarse-to-Fine Disentanglement of Intracranial Neural Dynamicscited by 1
  16. 2025Bridging Brain with Foundation Models through Self-Supervised Learningcited by 1
  17. 2025Region-Aware Reconstruction Strategy for Pre-training fMRI Foundation Modelcited by 1
  18. 2026Can Natural Image Autoencoders Compactly Tokenize fMRI Volumes for Long-Range Dynamics Modeling?cited by 1
  19. 2026Pretraining Induces a Reusable Spectral Basis for Downstream Task Adaptationcited by 1
  20. 2026From Clever Hans to Scientific Discovery: Interpreting EEG Foundational Transformers with LRPcited by 1
  21. 2026The Variance Brain Foundation Models Forgot: Third-Order Statistics Predict Cognition Where Billion-Parameter Models Failcited by 1
  22. 2026Beyond Single-Source Cognitive Taskonomy:Multi-Source Task Relations through fMRI Transfer Learningcited by 1
  23. 2026BrainFIBRE: A Foundation Model via Information Decomposition for Brain Microstructurecited by 1
  24. 2026Cross-Subject Modeling for Widefield Calcium Imaging via Atlas-Aligned Spatiotemporal Tokenizationcited by 1
  25. 2025Mapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging datacited by 0
  26. 2025BrainSymphony: A parameter-efficient multimodal foundation model for brain dynamics with limited datacited by 0
  27. 2025Introducing Multimodal Paradigm for Learning Sleep Staging PSG via General-Purpose Modelcited by 0
  28. 2026The Pictorial Cortex: Zero-Shot Cross-Subject fMRI-to-Image Reconstruction via Compositional Latent Modelingcited by 0
  29. 2026RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecturecited by 0
  30. 2026CryoLVM: Self-supervised Learning from Cryo-EM Density Maps with Large Vision Modelscited by 0
  31. 2026Diffusion-Guided Pretraining for Brain Graph Foundation Modelscited by 0
  32. 2026GOT-JEPA: Generic Object Tracking with Model Adaptation and Occlusion Handling using Joint-Embedding Predictive Architecturecited by 0
  33. 2026Interpretable Cross-Network Attention for Resting-State fMRI Representation Learningcited by 0
  34. 2026Toward a Multi-View Brain Network Foundation Model: Cross-View Consistency Learning Across Arbitrary Atlasescited by 0
  35. 2026Diffusion MRI Transformer with a Diffusion Space Rotary Positional Embedding (D-RoPE)cited by 0
  36. 2026Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivitycited by 0
  37. 2026Inverting Foundation Models of Brain Function with Simulation-Based Inferencecited by 0
  38. 2026Rhamba: Region-Aware Hybrid Attention-Mamba Framework for Self-Supervised Learning in Resting-State fMRIcited by 0

50 most cited of 64 All 64 (JSON)