# CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decoding

arXiv: https://arxiv.org/abs/2506.23075 · HTML: https://mlsanity.com/paper/2506.23075/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 119 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [EEG-Based Emotion Recognition Using Regularized Graph Neural Networks](https://mlsanity.com/paper/1907.07835/) (2019)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [NeuroLM: A Universal Multi-task Foundation Model for Bridging the Gap between Language and EEG Signals](https://mlsanity.com/paper/2409.00101/) (2024)
- [SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification](https://mlsanity.com/paper/2105.11043/) (2021)
- [Neuro-GPT: Towards A Foundation Model for EEG](https://mlsanity.com/paper/2311.03764/) (2023)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [Transformer Convolutional Neural Networks for Automated Artifact Detection in Scalp EEG](https://mlsanity.com/paper/2208.02405/) (2022)
- [Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification](https://mlsanity.com/paper/2405.19363/) (2024)
- [EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs](https://mlsanity.com/paper/2402.17772/) (2024)
- [Automatic Sleep Staging of EEG Signals: Recent Development, Challenges, and Future Directions](https://mlsanity.com/paper/2111.08446/) (2021)
- [EEG based Emotion Recognition: A Tutorial and Review](https://mlsanity.com/paper/2203.11279/) (2022)
- [Inter-database validation of a deep learning approach for automatic sleep scoring](https://mlsanity.com/paper/2009.10365/) (2020)
- [FlexCAD: Unified and Versatile Controllable CAD Generation with Fine-tuned Large Language Models](https://mlsanity.com/paper/2411.05823/) (2024)

## Built on by (most cited first, 30 of 53)

- [A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers](https://mlsanity.com/paper/2508.21148/) (2025)
- [EEG-FM-Bench: A Comprehensive Benchmark for the Systematic Evaluation and Diagnostic Analyses of EEG Foundation Models](https://mlsanity.com/paper/2508.17742/) (2025)
- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [Neural Codecs as Biosignal Tokenizers](https://mlsanity.com/paper/2510.09095/) (2025)
- [Uni-NTFM: A Unified Foundation Model for EEG Signal Representation Learning](https://mlsanity.com/paper/2509.24222/) (2025)
- [SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning](https://mlsanity.com/paper/2508.10298/) (2025)
- [DeeperBrain: A Neuro-Grounded EEG Foundation Model Towards Universal BCI](https://mlsanity.com/paper/2601.06134/) (2026)
- [ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Models](https://mlsanity.com/paper/2509.22556/) (2025)
- [Let EEG Models Learn EEG](https://mlsanity.com/paper/2605.21280/) (2026)
- [MIRepNet: A Pipeline and Foundation Model for EEG-Based Motor Imagery Classification](https://mlsanity.com/paper/2507.20254/) (2025)
- [NeuroWeaver: An Autonomous Evolutionary Agent for Exploring the Programmatic Space of EEG Analysis Pipelines](https://mlsanity.com/paper/2602.13473/) (2026)
- [SingLEM: Single-Channel Large EEG Model](https://mlsanity.com/paper/2509.17920/) (2025)
- [What Do EEG Foundation Models Capture from Human Brain Signals?](https://mlsanity.com/paper/2605.11410/) (2026)
- [Brain-OF: An Omnifunctional Foundation Model for fMRI, EEG and MEG](https://mlsanity.com/paper/2602.23410/) (2026)
- [Same Brain, Different Prediction: How Preprocessing Choices Undermine EEG Decoding Reliability](https://mlsanity.com/paper/2605.07212/) (2026)
- [KAST-BAR: Knowledge-Anchored Semantically-Dynamic Topology Brain Autoregressive Modeling for Universal Neural Interpretation](https://mlsanity.com/paper/2605.13133/) (2026)
- [SCOPE: Structured Prototype-Guided Adaptation for EEG Foundation Models with Limited Labels](https://mlsanity.com/paper/2602.17251/) (2026)
- [SpecMoE: Spectral Mixture-of-Experts Foundation Model for Cross-Species EEG Decoding](https://mlsanity.com/paper/2603.16739/) (2026)
- [CLEF: EEG Foundation Model for Learning Clinical Semantics](https://mlsanity.com/paper/2605.10817/) (2026)
- [A Multi-dimensional Framework for Evaluating Generalization in EEG Foundation Models](https://mlsanity.com/paper/2605.28563/) (2026)
- [MANAS-2: Constrained Reconstruction for EEG Foundation Models](https://mlsanity.com/paper/2609.13717/) (2026)
- [A Systematic Evaluation of Self-Supervised Learning for Label-Efficient Sleep Staging with Wearable EEG](https://mlsanity.com/paper/2510.07960/) (2025)
- [Fusion of Multiscale Features Via Centralized Sparse-attention Network for EEG Decoding](https://mlsanity.com/paper/2512.18689/) (2025)
- [NeuroFlow: Toward Unified Visual Encoding and Decoding from Neural Activity](https://mlsanity.com/paper/2604.09817/) (2026)
- [A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection](https://mlsanity.com/paper/2604.26379/) (2026)
- [MSCGC-KAN: Multi-scale Causal Graph Convolution and KAN-inspired Analytic-basis Mapping for EEG Emotion Recognition](https://mlsanity.com/paper/2605.26624/) (2026)
- [Context-aware tokenization for Cross-subject Emotion Decoding from EEG](https://mlsanity.com/paper/2606.00884/) (2026)
- [EvoBrain: Continual Learning of EEG Foundation Models Across Heterogeneous BCI Tasks](https://mlsanity.com/paper/2606.01767/) (2026)
- [Channel-Preserving Representation Alignment for EEG-to-Music Reconstruction](https://mlsanity.com/paper/2606.04040/) (2026)
- [RECTOR: Masked Region-Channel-Temporal Modeling for Affective and Cognitive Representation Learning](https://mlsanity.com/paper/2606.15278/) (2026)

## Abstract (arXiv)

Understanding and decoding brain activity from electroencephalography (EEG) signals is a fundamental challenge in neuroscience and AI, with applications in cognition, emotion recognition, diagnosis, and brain-computer interfaces. While recent EEG foundation models advance generalized decoding via unified architectures and large-scale pretraining, they adopt a scale-agnostic dense modeling paradigm inherited from NLP and vision. This design neglects a core property of neural activity: cross-scale spatiotemporal structure. EEG task patterns span a wide range of temporal and spatial scales, from short bursts to slow rhythms, and from localized cortical responses to distributed interactions. Ignoring this diversity leads to suboptimal representations and weak generalization. We propose CSBrain, a Cross-scale Spatiotemporal Brain foundation model for generalized EEG decoding. CSBrain introduces: (i) Cross-scale Spatiotemporal Tokenization (CST), which aggregates multi-scale features from localized temporal windows and anatomical brain regions into compact scale-aware tokens; and (ii) Structured Sparse Attention (SSA), which captures cross-window and cross-region dependencies, enhancing scale diversity while removing spurious correlations. CST and SSA are alternately stacked to progressively integrate multi-scale dependencies. Experiments on 11 EEG tasks across 16 datasets show that CSBrain consistently outperforms task-specific and foundation model baselines. These results establish cross-scale modeling as a key inductive bias and position CSBrain as a robust backbone for future brain-AI research.