DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations
Not on the map. Why?
Abstract (arXiv)
Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time attention, spatio-temporal resampling, input-conditioned positional embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 26 of its 58 references that are in the map. First posted on arXiv 22 Dec 2025.
Builds on
26 of 58 references.
References · 26
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
- 2020Quantifying Attention Flow in Transformerscited by 759
- 2024Unified Training of Universal Time Series Forecasting Transformerscited by 588
- 2023Scaling Data-Constrained Language Modelscited by 391
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 311
- 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 188
14 more
- 2024How to Train Long-Context Language Models (Effectively)cited by 141
- 2023BrainBERT: Self-supervised representation learning for intracranial recordingscited by 70
- 2023Neuro-GPT: Towards A Foundation Model for EEGcited by 69
- 2024Chinchilla Scaling: A replication attemptcited by 65
- 2024EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Modelcited by 51
- 2024Population Transformer: Learning Population-level Representations of Neural Activitycited by 26
- 2025Scaling laws for decoding images from brain activitycited by 21
- 2024Brain Treebank: Large-scale intracranial recordings from naturalistic language stimulicited by 15
- 2024Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signalscited by 12
- 2025BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activitycited by 7
- 2025Is Limited Participant Diversity Impeding EEG-based Machine Learning?cited by 6
- 2025A foundation model with multi-variate parallel attention to generate neuronal activitycited by 6
- 2025Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimulicited by 3
- 2025When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Modelscited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Q-DIVER: Integrated Quantum Transfer Learning and Differentiable Quantum Architecture Search with EEG Datacited by 0
- 2026Pretraining for Sample-Efficient Neural Interfacescited by 0
- 2026iMINDBench: iEEG Multi-Institution Neural Decoding Benchmarkcited by 0
- 2026BAT-CLIP: Trimodal Alignment of Brain, Audio and Textcited by 0
- 2026BAT-CLIP: Trimodal Alignment of Brain, Audio and Text25 Sep 2026
- 2026iMINDBench: iEEG Multi-Institution Neural Decoding Benchmark16 Sep 2026
- 2026Pretraining for Sample-Efficient Neural Interfaces11 Sep 2026
- 2026Q-DIVER: Integrated Quantum Transfer Learning and Differentiable Quantum Architecture Search with EEG Data30 Mar 2026