# Misconduct in Post-Selections and Deep Learning

arXiv: https://arxiv.org/abs/2403.00773 · HTML: https://mlsanity.com/paper/2403.00773/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 38 references that are in the map.
- First posted on arXiv 13 Feb 2024.

## Builds on (most cited first, 5 of 5)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization](https://mlsanity.com/paper/1909.13698/) (2019)
- [Why Deep Learning's Performance Data Are Misleading](https://mlsanity.com/paper/2208.11228/) (2022)
- [On "Deep Learning" Misconduct](https://mlsanity.com/paper/2211.16350/) (2022)

## Abstract (arXiv)

This is a theoretical paper on "Deep Learning" misconduct in particular and Post-Selection in general. As far as the author knows, the first peer-reviewed papers on Deep Learning misconduct are [32], [37], [36]. Regardless of learning modes, e.g., supervised, reinforcement, adversarial, and evolutional, almost all machine learning methods (except for a few methods that train a sole system) are rooted in the same misconduct -- cheating and hiding -- (1) cheating in the absence of a test and (2) hiding bad-looking data. It was reasoned in [32], [37], [36] that authors must report at least the average error of all trained networks, good and bad, on the validation set (called general cross-validation in this paper). Better, report also five percentage positions of ranked errors. From the new analysis here, we can see that the hidden culprit is Post-Selection. This is also true for Post-Selection on hand-tuned or searched hyperparameters, because they are random, depending on random observation data. Does cross-validation on data splits rescue Post-Selections from the Misconducts (1) and (2)? The new result here says: No. Specifically, this paper reveals that using cross-validation for data splits is insufficient to exonerate Post-Selections in machine learning. In general, Post-Selections of statistical learners based on their errors on the validation set are statistically invalid.