# The Role of Review Process Failures in Affective State Estimation: An Empirical Investigation of DEAP Dataset

arXiv: https://arxiv.org/abs/2508.02417 · HTML: https://mlsanity.com/paper/2508.02417/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 145 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans](https://mlsanity.com/paper/2008.06388/) (2020)
- [Dos and Don'ts of Machine Learning in Computer Security](https://mlsanity.com/paper/2010.09470/) (2020)
- [TSception: Capturing Temporal Dynamics and Spatial Asymmetry from EEG for Emotion Recognition](https://mlsanity.com/paper/2104.02935/) (2021)
- [Identifying Stable Patterns over Time for Emotion Recognition from EEG](https://mlsanity.com/paper/1601.02197/) (2016)
- [Data Augmentation for Enhancing EEG-based Emotion Recognition with Deep Generative Models](https://mlsanity.com/paper/2006.05331/) (2020)
- [4D Attention-based Neural Network for EEG Emotion Recognition](https://mlsanity.com/paper/2101.05484/) (2021)
- [Subject Independent Emotion Recognition using EEG Signals Employing Attention Driven Neural Networks](https://mlsanity.com/paper/2106.03461/) (2021)
- [Beware of Overestimated Decoding Performance Arising from Temporal Autocorrelations in Electroencephalogram Signals](https://mlsanity.com/paper/2405.17024/) (2024)
- [Affective EEG-Based Person Identification Using the Deep Learning Approach](https://mlsanity.com/paper/1807.03147/) (2018)
- [ScalingNet: extracting features from raw EEG data for emotion recognition](https://mlsanity.com/paper/2105.13987/) (2021)
- [Towards emotion recognition for virtual environments: an evaluation of EEG features on benchmark dataset](https://mlsanity.com/paper/2210.13876/) (2022)
- [Automated Feature Extraction on AsMap for Emotion Classification using EEG](https://mlsanity.com/paper/2201.12055/) (2022)

## Abstract (arXiv)

The reliability of affective state estimation using EEG data is in question, given the variability in reported performance and the lack of standardized evaluation protocols. To investigate this, we reviewed 101 studies, focusing on the widely used DEAP dataset for emotion recognition. Our analysis revealed widespread methodological issues that include data leakage from improper segmentation, biased feature selection, flawed hyperparameter optimization, neglect of class imbalance, and insufficient methodological reporting. Notably, we found that nearly 87% of the reviewed papers contained one or more of these errors. Moreover, through experimental analysis, we observed that such methodological flaws can inflate the classification accuracy by up to 46%. These findings reveal fundamental gaps in standardized evaluation practices and highlight critical deficiencies in the peer review process for machine learning applications in neuroscience, emphasizing the urgent need for stricter methodological standards and evaluation protocols.