# EEG-DLite: Dataset Distillation for Efficient Large EEG Model Training

arXiv: https://arxiv.org/abs/2512.12210 · HTML: https://mlsanity.com/paper/2512.12210/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 13 Dec 2025.

## Builds on (most cited first, 12 of 12)

- [Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCI](https://mlsanity.com/paper/2405.18765/) (2024)
- [GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training](https://mlsanity.com/paper/2103.00123/) (2021)
- [CBraMod: A Criss-Cross Brain Foundation Model for EEG Decoding](https://mlsanity.com/paper/2412.07236/) (2024)
- [ICE: Inter-instance Contrastive Encoding for Unsupervised Person Re-identification](https://mlsanity.com/paper/2103.16364/) (2021)
- [M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy](https://mlsanity.com/paper/2312.15927/) (2023)
- [Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer](https://mlsanity.com/paper/2205.09928/) (2022)
- [Brain Foundation Models: A Survey on Advancements in Neural Signal Processing and Brain Discovery](https://mlsanity.com/paper/2503.00580/) (2025)
- [An Incremental Dimensionality Reduction Method for Visualizing Streaming Multidimensional Data](https://mlsanity.com/paper/1905.04000/) (2019)
- [Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networks](https://mlsanity.com/paper/2410.02116/) (2024)
- [Artifact Detection and Correction in EEG data: A Review](https://mlsanity.com/paper/2106.13081/) (2021)
- [The Case for Cleaner Biosignals: High-fidelity Neural Compressor Enables Transfer from Cleaner iEEG to Noisier EEG](https://mlsanity.com/paper/2502.17462/) (2025)
- [How Much Data is Enough? Optimization of Data Collection for Artifact Detection in EEG Recordings](https://mlsanity.com/paper/2411.11886/) (2024)

## Abstract (arXiv)

Large-scale EEG foundation models have shown strong generalization across a range of downstream tasks, but their training remains resource-intensive due to the volume and variable quality of EEG data. In this work, we introduce EEG-DLite, a data distillation framework that enables more efficient pre-training by selectively removing noisy and redundant samples from large EEG datasets. EEG-DLite begins by encoding EEG segments into compact latent representations using a self-supervised autoencoder, allowing sample selection to be performed efficiently and with reduced sensitivity to noise. Based on these representations, EEG-DLite filters out outliers and minimizes redundancy, resulting in a smaller yet informative subset that retains the diversity essential for effective foundation model training. Through extensive experiments, we demonstrate that training on only 5 percent of a 2,500-hour dataset curated with EEG-DLite yields performance comparable to, and in some cases better than, training on the full dataset across multiple downstream tasks. To our knowledge, this is the first systematic study of pre-training data distillation in the context of EEG foundation models. EEG-DLite provides a scalable and practical path toward more effective and efficient physiological foundation modeling. The code is available at https://github.com/t170815518/EEG-DLite.