mlsanityData to 7 Oct 2026

DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations

arXiv 2512.19097 ↗First posted 22 Dec 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time attention, spatio-temporal resampling, input-conditioned positional embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 58 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

26 of 58 references.

References · 26

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  7. 2020Quantifying Attention Flow in Transformerscited by 759
  8. 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
  9. 2023Scaling Data-Constrained Language Modelscited by 391
  10. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
  11. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  12. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 188
14 more
  1. 2024How to Train Long-Context Language Models (Effectively)cited by 141
  2. 2023BrainBERT: Self-supervised representation learning for intracranial recordingscited by 70
  3. 2023Neuro-GPT: Towards A Foundation Model for EEGcited by 69
  4. 2024Chinchilla Scaling: A replication attemptcited by 65
  5. 2024EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Modelcited by 51
  6. 2024Population Transformer: Learning Population-level Representations of Neural Activitycited by 26
  7. 2025Scaling laws for decoding images from brain activitycited by 21
  8. 2024Brain Treebank: Large-scale intracranial recordings from naturalistic language stimulicited by 15
  9. 2024Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signalscited by 12
  10. 2025BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activitycited by 7
  11. 2025Is Limited Participant Diversity Impeding EEG-based Machine Learning?cited by 6
  12. 2025A foundation model with multi-variate parallel attention to generate neuronal activitycited by 6
  13. 2025Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimulicited by 3
  14. 2025When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.