mlsanityData to 14 Oct 2026

What masking geometry works best for EEG foundation models?

arXiv 2609.33487 ↗First posted 27 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

EEG foundation models hold promise for scalable brain-signal decoding across clinical and cognitive neuroscience applications, yet their pre-training pipelines remain poorly understood. Among design choices, the masking strategy is particularly critical: it determines what the network must predict and from which context. Yet it has never been ablated in isolation, as each new model bundles a new masking strategy with a new backbone and objective. In this paper, we formalize the design choices for spatio-temporal masking strategies and train various models with a single pipeline under varying masking configurations across two SSL frameworks (MAE and JEPA). We then systematically evaluate the resulting 58 pre-trained models on the 12 datasets of OpenEEGBench under a linear probe. Both frameworks agree on an optimal masking configuration and on shared failure modes. Outside these, performance is robust: 11 MAE and 9 JEPA configurations are statistically indistinguishable from the best. We further identify a novel JEPA-specific failure mode, tagged bias-inflation collapse, invisible to standard detectors. With a well-chosen mask, our pipeline reaches REVE-level downstream performance at a fraction of REVE's pre-training compute.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 105 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

35 of 105 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2020GLU Variants Improve Transformercited by 1,804
  5. 2019Root Mean Square Layer Normalizationcited by 1,480
  6. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  7. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  8. 2021Scaling Vision Transformerscited by 999
  9. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  10. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  11. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  12. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
23 more
  1. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  2. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  3. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
  4. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  5. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 199
  6. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
  7. 2022RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rankcited by 153
  8. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  9. 2025REVE: A Foundation Model for EEG -- Adapting to Any Setup with Large-Scale Pretraining on 25,000 Subjectscited by 80
  10. 2023Neuro-GPT: Towards A Foundation Model for EEGcited by 74
  11. 2023BrainBERT: Self-supervised representation learning for intracranial recordingscited by 72
  12. 2025CSBrain: A Cross-scale Spatiotemporal Brain Foundation Model for EEG Decodingcited by 53
  13. 2024EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputscited by 50
  14. 2022Understanding Collapse in Non-Contrastive Siamese Representation Learningcited by 42
  15. 2025Joint Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self Supervised Learningcited by 36
  16. 2016A Multimodal Approach to Estimating Vigilance Using EEG and Forehead EOGcited by 33
  17. 2025LUNA: Efficient and Topology-Agnostic Foundation Model for EEG Signal Analysiscited by 26
  18. 2024PCP-MAE: Learning to Predict Centers for Point Masked Autoencoderscited by 24
  19. 2024Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learningcited by 20
  20. 2024S-JEPA: towards seamless cross-dataset transfer through dynamic spatial attentioncited by 16
  21. 2026NeuralBench: A Unifying Framework to Benchmark NeuroAI Modelscited by 16
  22. 2023Stochastic positional embeddings improve masked image modelingcited by 12
  23. 2015Using Riemannian geometry for SSVEP-based Brain Computer Interfacecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SpecMoE: Spectral Mixture-of-Experts Foundation Model for Cross-Species EEG Decoding
  2. 2026Pretrained, Frozen, Still Leaking: Auditing Cross-Encoder Attribute Transfer in EEG Foundation Models
  3. 2026Aperiodic and Low-Frequency Spectral Bias in Reconstruction based EEG Foundation Models
  4. 2025NeurIPT: Foundation Model for Neural Interfaces
  5. 2026PRISM: Exploring Heterogeneous Pretrained EEG Foundation Model Transfer to Clinical Differential Diagnosis
  6. 2026Representational and Functional Robustness to Electrode Montages in EEG Foundation Models
  7. 2026DLink: Distilling Layer-wise and Dominant Knowledge from EEG Foundation Models
  8. 2026Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders