# DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations

arXiv: https://arxiv.org/abs/2512.19097 · HTML: https://mlsanity.com/paper/2512.19097/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 58 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Builds on (most cited first, 26 of 26)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Unified Training of Universal Time Series Forecasting Transformers](https://mlsanity.com/paper/2402.02592/) (2024)
- [Scaling Data-Constrained Language Models](https://mlsanity.com/paper/2305.16264/) (2023)
- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer](https://mlsanity.com/paper/2203.03466/) (2022)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [How to Train Long-Context Language Models (Effectively)](https://mlsanity.com/paper/2410.02660/) (2024)
- [Neuro-GPT: Towards A Foundation Model for EEG](https://mlsanity.com/paper/2311.03764/) (2023)
- [BrainBERT: Self-supervised representation learning for intracranial recordings](https://mlsanity.com/paper/2302.14367/) (2023)
- [Chinchilla Scaling: A replication attempt](https://mlsanity.com/paper/2404.10102/) (2024)
- [EEGFormer: Towards Transferable and Interpretable Large-Scale EEG Foundation Model](https://mlsanity.com/paper/2401.10278/) (2024)
- [Population Transformer: Learning Population-level Representations of Neural Activity](https://mlsanity.com/paper/2406.03044/) (2024)
- [Scaling laws for decoding images from brain activity](https://mlsanity.com/paper/2501.15322/) (2025)
- [Brain Treebank: Large-scale intracranial recordings from naturalistic language stimuli](https://mlsanity.com/paper/2411.08343/) (2024)
- [Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals](https://mlsanity.com/paper/2405.11459/) (2024)
- [BaRISTA: Brain Scale Informed Spatiotemporal Representation of Human Intracranial Neural Activity](https://mlsanity.com/paper/2512.12135/) (2025)
- [Is Limited Participant Diversity Impeding EEG-based Machine Learning?](https://mlsanity.com/paper/2503.13497/) (2025)
- [A foundation model with multi-variate parallel attention to generate neuronal activity](https://mlsanity.com/paper/2506.20354/) (2025)
- [Neuroprobe: Evaluating Intracranial Brain Responses to Naturalistic Stimuli](https://mlsanity.com/paper/2509.21671/) (2025)
- [When Long Helps Short: How Context Length in Supervised Fine-tuning Affects Behavior of Large Language Models](https://mlsanity.com/paper/2509.18762/) (2025)

## Built on by (most cited first, 5 of 5)

- [Q-DIVER: Integrated Quantum Transfer Learning and Differentiable Quantum Architecture Search with EEG Data](https://mlsanity.com/paper/2603.28122/) (2026)
- [Pretraining for Sample-Efficient Neural Interfaces](https://mlsanity.com/paper/2609.13507/) (2026)
- [iMINDBench: iEEG Multi-Institution Neural Decoding Benchmark](https://mlsanity.com/paper/2609.18104/) (2026)
- [BAT-CLIP: Trimodal Alignment of Brain, Audio and Text](https://mlsanity.com/paper/2609.31180/) (2026)
- [Training and Scaling Compute-Optimal Physiological Waveform Foundation Models](https://mlsanity.com/paper/2610.05649/) (2026)

## Abstract (arXiv)

Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time attention, spatio-temporal resampling, input-conditioned positional embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link.