# The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study

arXiv: https://arxiv.org/abs/2505.13021 · HTML: https://mlsanity.com/paper/2505.13021/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 58 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces](https://mlsanity.com/paper/1611.08024/) (2016)
- [Deep learning-based electroencephalography analysis: a systematic review](https://mlsanity.com/paper/1901.05498/) (2019)
- [Metrics for Multi-Class Classification: an Overview](https://mlsanity.com/paper/2008.05756/) (2020)
- [TorchAudio: Building Blocks for Audio and Speech Processing](https://mlsanity.com/paper/2110.15018/) (2021)
- [Subject-Aware Contrastive Learning for Biosignals](https://mlsanity.com/paper/2007.04871/) (2020)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [ICLabel: An automated electroencephalographic independent component classifier, dataset, and website](https://mlsanity.com/paper/1901.07915/) (2019)
- [Machine Learning for Predicting Epileptic Seizures Using EEG Signals: A Review](https://mlsanity.com/paper/2002.01925/) (2020)
- [Self-supervised Contrastive Learning for EEG-based Sleep Staging](https://mlsanity.com/paper/2109.07839/) (2021)
- [Task-oriented Self-supervised Learning for Anomaly Detection in Electroencephalography](https://mlsanity.com/paper/2207.01391/) (2022)
- [Motor Imagery Decoding Using Ensemble Curriculum Learning and Collaborative Training](https://mlsanity.com/paper/2211.11460/) (2022)
- [SelfEEG: A Python library for Self-Supervised Learning in Electroencephalography](https://mlsanity.com/paper/2401.05405/) (2023)
- [The more, the better? Evaluating the role of EEG preprocessing for deep learning applications](https://mlsanity.com/paper/2411.18392/) (2024)
- [Deep comparisons of Neural Networks from the EEGNet family](https://mlsanity.com/paper/2302.08797/) (2023)
- [MP-SeizNet: A Multi-Path CNN Bi-LSTM Network for Seizure-Type Classification Using EEG](https://mlsanity.com/paper/2211.04628/) (2022)

## Built on by (most cited first, 4 of 4)

- [TransformEEG: Towards Improving Model Generalizability in Deep Learning-based EEG Parkinson's Disease Detection](https://mlsanity.com/paper/2507.07622/) (2025)
- [MultiDiffNet: A Multi-Objective Diffusion Framework for Generalizable Brain Decoding](https://mlsanity.com/paper/2511.18294/) (2025)
- [Interpretable EEG biomarkers with bag-of-waves: Spatial and temporal waveform dictionaries for low-data regimes](https://mlsanity.com/paper/2607.22508/) (2026)
- [Checkpoint Selection and Evaluation in EEG Emotion Recognition](https://mlsanity.com/paper/2607.27655/) (2026)

## Abstract (arXiv)

Deep learning is significantly advancing the analysis of electroencephalography (EEG) data by effectively discovering highly nonlinear patterns within the signals. Data partitioning and cross-validation are crucial for assessing model performance and ensuring study comparability, as they can produce varied results and data leakage due to specific signal properties (e.g., biometric). Such variability leads to incomparable studies and, increasingly, overestimated performance claims, which are detrimental to the field. Nevertheless, no comprehensive guidelines for proper data partitioning and cross-validation exist in the domain, nor is there a quantitative evaluation of their impact on model accuracy, reliability, and generalizability. To assist researchers in identifying optimal experimental strategies, this paper thoroughly investigates the role of data partitioning and cross-validation in evaluating EEG deep learning models. Five cross-validation settings are compared across three supervised cross-subject classification tasks (BCI, Parkinson's, and Alzheimer's disease detection) and four established architectures of increasing complexity (ShallowConvNet, EEGNet, DeepConvNet, and Temporal-based ResNet). The comparison of over 100,000 trained models underscores, first, the importance of using subject-based cross-validation strategies for evaluating EEG deep learning models, except when within-subject analyses are acceptable (e.g., BCI). Second, it highlights the greater reliability of nested approaches (N-LNSO) compared to non-nested counterparts, which are prone to data leakage and favor larger models overfitting to validation data. In conclusion, this work provides EEG deep learning researchers with an analysis of data partitioning and cross-validation and offers guidelines to avoid data leakage, currently undermining the domain with potentially overestimated performance claims.