# EEG-D3: A Solution to the Hidden Overfitting Problem of Deep Learning Models

arXiv: https://arxiv.org/abs/2512.13806 · HTML: https://mlsanity.com/paper/2512.13806/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 15 Dec 2025.

## Builds on (most cited first, 13 of 13)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Uncovering the structure of clinical EEG signals with self-supervised learning](https://mlsanity.com/paper/2007.16104/) (2020)
- [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series](https://mlsanity.com/paper/1707.03321/) (2017)
- [Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learning](https://mlsanity.com/paper/2303.16535/) (2023)
- [2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets](https://mlsanity.com/paper/2202.12950/) (2022)
- [Latent Alignment with Deep Set EEG Decoders](https://mlsanity.com/paper/2311.17968/) (2023)
- [EEGminer: Discovering Interpretable Features of Brain Activity with Learnable Filters](https://mlsanity.com/paper/2110.10009/) (2021)
- [Representation learning for improved interpretability and classification accuracy of clinical factors from EEG](https://mlsanity.com/paper/2010.15274/) (2020)
- [Advancing Brainwave Modeling with a Codebook-Based Foundation Model](https://mlsanity.com/paper/2505.16724/) (2025)

## Abstract (arXiv)

Deep learning for decoding EEG signals has gained traction, with many claims to state-of-the-art accuracy. However, despite the convincing benchmark performance, successful translation to real applications is limited. The frequent disconnect between performance on controlled BCI benchmarks and its lack of generalisation to practical settings indicates hidden overfitting problems. We introduce Disentangled Decoding Decomposition (D3), a weakly supervised method for training deep learning models across EEG datasets. By predicting the place in the respective trial sequence from which the input window was sampled, EEG-D3 separates latent components of brain activity, akin to non-linear ICA. We utilise a novel model architecture with fully independent sub-networks for strict interpretability. We outline a feature interpretation paradigm to contrast the component activation profiles on different datasets and inspect the associated temporal and spatial filters. The proposed method reliably separates latent components of brain activity on motor imagery data. Training downstream classifiers on an appropriate subset of these components prevents hidden overfitting caused by task-correlated artefacts, which severely affects end-to-end classifiers. We further exploit the linearly separable latent space for effective few-shot learning on sleep stage classification. The ability to distinguish genuine components of brain activity from spurious features results in models that avoid the hidden overfitting problem and generalise well to real-world applications, while requiring only minimal labelled data. With interest to the neuroscience community, the proposed method gives researchers a tool to separate individual brain processes and potentially even uncover heretofore unknown dynamics.