# SAMBA: Toward a Long-Context EEG Foundation Model via Spatial Embedding and Differential Mamba

arXiv: https://arxiv.org/abs/2511.18571 · HTML: https://mlsanity.com/paper/2511.18571/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Physiological signal modeling, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 14 of its 46 references that are in the map.
- First posted on arXiv 23 Nov 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Physiological signal modeling](https://mlsanity.com/ssm/physiological-signal-modeling/)

## Builds on (most cited first, 14 of 14)

- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data](https://mlsanity.com/paper/2101.12037/) (2021)
- [Zamba: A Compact 7B SSM Hybrid Model](https://mlsanity.com/paper/2405.16712/) (2024)
- [Transformer-based Spatial-Temporal Feature Learning for EEG Decoding](https://mlsanity.com/paper/2106.11170/) (2021)
- [MOABB: Trustworthy algorithm benchmarking for BCIs](https://mlsanity.com/paper/1805.06427/) (2018)
- [MAEEG: Masked Auto-encoder for EEG Representation Learning](https://mlsanity.com/paper/2211.02625/) (2022)
- [Transformer Convolutional Neural Networks for Automated Artifact Detection in Scalp EEG](https://mlsanity.com/paper/2208.02405/) (2022)
- [EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputs](https://mlsanity.com/paper/2402.17772/) (2024)
- [Efficient Long Sequence Modeling via State Space Augmented Transformer](https://mlsanity.com/paper/2212.08136/) (2022)
- [A Multimodal Approach to Estimating Vigilance Using EEG and Forehead EOG](https://mlsanity.com/paper/1611.08492/) (2016)
- [Self-supervised EEG Representation Learning for Automatic Sleep Staging](https://mlsanity.com/paper/2110.15278/) (2021)

## Built on by (most cited first, 4 of 4)

- [Brain4FMs: A Benchmark of Foundation Models for Electrical Brain Signal](https://mlsanity.com/paper/2602.11558/) (2026)
- [Representational and Functional Robustness to Electrode Montages in EEG Foundation Models](https://mlsanity.com/paper/2609.36288/) (2026)
- [NeurDuo-EEG: A Long-Sequence EEG Foundation Model with Persistent State and Explicit Memory](https://mlsanity.com/paper/2609.38587/) (2026)
- [CortexBridge: Cortical Alignment of EEG Montages for Foundation Models](https://mlsanity.com/paper/2610.01124/) (2026)

## Abstract (arXiv)

Long-sequence electroencephalogram (EEG) modeling is essential for developing generalizable EEG representation models. This need arises from the high sampling rate of EEG data and the long recording durations required to capture extended neurological patterns in brain activity. Transformer-based models have shown promise in modeling short sequences of a few seconds; however, their quadratic complexity limits scalability to longer contexts. Moreover, variability in electrode montage across available datasets, along with inter-subject differences in brain signals, pose significant challenges to developing a generalizable and robust foundation model. We propose textitSAMBA, a self-supervised learning framework with a Mamba-based U-shaped encoder-decoder architecture, which effectively captures long-range temporal dependencies and spatial variability in EEG data. Leveraging the inherent ability of Mamba in processing long context sizes, we introduce: (1) textitTemporal Semantic Random Masking for semantic-level sequence reconstruction, (2) a textitMulti-Head Differential Mamba module to suppress redundancy and emphasize salient temporal structures, and (3) a textitSpatial-Adaptive Input Embedding that learns unified embeddings in a three-dimensional Euclidean space, enabling robustness across devices. Experiments on thirteen EEG datasets across diverse tasks, electrode configurations, and sequence durations demonstrate that SAMBA consistently outperforms state-of-the-art methods while maintaining low memory consumption and inference time. We also show the learned spatial weight maps from our embedding module align closely with task-relevant neurophysiological regions, demonstrating the learnability and interpretability of SAMBA. These results highlight SAMBA's scalability and practical potential as a foundation model for real-time brain-computer interface applications.