Neural State Prediction: Obstructing Shortcut Learning in EEG Foundation Models
Not on the map. Why?
Abstract (arXiv)
EEG foundation models increasingly use masked prediction to learn from unlabeled recordings, but optimizing this objective does not ensure transferable neural representations. A central challenge is that stable positional cues and local correlations can make masked regions predictable without integrating distributed neural context. To reduce this reliance on low-information prediction paths, we introduce Neural State Prediction (NSP), a latent-predictive framework that constrains both the prediction target and the available context. NSP uses a Target Encoder updated by an exponential moving average (EMA) to define latent supervision. Identity residualization removes additive effects associated with channel identity and relative time from the targets, while topology-separated context excludes their immediate spatial and temporal neighborhood from the visible input. We pretrain NSP on 2.2 million EEG segments from TUEG and evaluate it across 30 downstream datasets spanning clinical diagnosis, sleep staging, emotion recognition, motor imagery, event-related potentials, cognitive-state decoding, and language retrieval. Under full-parameter multi-task fine-tuning on EEG-FM-Bench, NSP achieves 63.94 macro balanced accuracy across 14 datasets, exceeding the strongest evaluated baseline by 2.35 percentage points. Controlled component ablations assess the contribution of each mechanism, while matched context controls and held-out interventions characterize the role of context geometry, signal content, and positional information. Jointly designing latent targets and their context offers a promising direction for EEG foundation models that learn from distributed signal structure.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 72 references that are in the map. First posted on arXiv 25 Sep 2026.
Builds on
33 of 72 references.
References · 33
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
- 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 628
21 more
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
- 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
- 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 199
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
- 2021Transformer-based Spatial-Temporal Feature Learning for EEG Decodingcited by 97
- 2025REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjectscited by 80
- 2024Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Maskingcited by 65
- 2025CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decodingcited by 53
- 2024EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputscited by 50
- 2020Inter-database validation of a deep learning approach for automatic sleep scoringcited by 38
- 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 36
- 2022Data augmentation for learning predictive models on EEG: a systematic comparisoncited by 33
- 2018Improved EEG Event Classification Using Differential Energycited by 32
- 2025CodeBrain: Bridging Decoupled Tokenizer and Multi-Scale Architecture for EEG Foundation Modelcited by 28
- 2025LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysiscited by 26
- 2024S-JEPA: towards seamless cross-dataset transfer through dynamic spatial attentioncited by 16
- 2026NeuralBench: A Unifying Framework to Benchmark NeuroAI Modelscited by 16
- 2025THD-BAR: Topology Hierarchical Derived Brain Autoregressive Modeling for EEG Generic Representationscited by 8
- 2025ECHO: Toward Contextual Seq2Seq Paradigms in Large EEG Modelscited by 7
- 2026Laya: A LeJEPA Approach to EEG via Latent Prediction over Reconstructioncited by 4
- 2026KAST-BAR: Knowledge-Anchored Semantically-Dynamic Topology Brain Autoregressive Modeling for Universal Neural Interpretationcited by 2
Built on by
Not cited yet.