# Classification at the Accuracy Limit -- Facing the Problem of Data Ambiguity

arXiv: https://arxiv.org/abs/2206.01922 · HTML: https://mlsanity.com/paper/2206.01922/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 48 references that are in the map.
- First posted on arXiv 4 Jun 2022.

## Builds on (most cited first, 9 of 9)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy](https://mlsanity.com/paper/1710.02094/) (2017)
- [How deep is deep enough? -- Quantifying class separability in the hidden layers of deep neural networks](https://mlsanity.com/paper/1811.01753/) (2018)
- [Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future](https://mlsanity.com/paper/2108.04543/) (2021)
- [A statistical method for analyzing and comparing spatiotemporal cortical activation patterns](https://mlsanity.com/paper/1611.07677/) (2016)
- [Analysis of structure and dynamics in three-neuron motifs](https://mlsanity.com/paper/1811.05225/) (2018)
- [Neural Network based Successor Representations of Space and Language](https://mlsanity.com/paper/2202.11190/) (2022)
- [Will we ever have Conscious Machines?](https://mlsanity.com/paper/2003.14132/) (2020)
- [Stochastic resonance controlled upregulation of internal noise after hearing loss as a putative correlate of tinnitus-related neuronal hyperactivity](https://mlsanity.com/paper/1603.04721/) (2016)

## Built on by (most cited first, 6 of 6)

- [Word class representations spontaneously emerge in a deep neural network trained on next word prediction](https://mlsanity.com/paper/2302.07588/) (2023)
- [Conceptual Cognitive Maps Formation with Neural Successor Networks and Word Embeddings](https://mlsanity.com/paper/2307.01577/) (2023)
- [Analysis of Argument Structure Constructions in a Deep Recurrent Language Model](https://mlsanity.com/paper/2408.03062/) (2024)
- [Analysis of Argument Structure Constructions in the Large Language Model BERT](https://mlsanity.com/paper/2408.04270/) (2024)
- [Analyzing Narrative Processing in Large Language Models (LLMs): Using GPT4 to test BERT](https://mlsanity.com/paper/2405.02024/) (2024)
- [Author-Specific Linguistic Patterns Unveiled: A Deep Learning Study on Word Class Distributions](https://mlsanity.com/paper/2501.10072/) (2025)

## Abstract (arXiv)

Data classification, the process of analyzing data and organizing it into categories, is a fundamental computing problem of natural and artificial information processing systems. Ideally, the performance of classifier models would be evaluated using unambiguous data sets, where the 'correct' assignment of category labels to the input data vectors is unequivocal. In real-world problems, however, a significant fraction of actually occurring data vectors will be located in a boundary zone between or outside of all categories, so that perfect classification cannot even in principle be achieved. We derive the theoretical limit for classification accuracy that arises from the overlap of data categories. By using a surrogate data generation model with adjustable statistical properties, we show that sufficiently powerful classifiers based on completely different principles, such as perceptrons and Bayesian models, all perform at this universal accuracy limit. Remarkably, the accuracy limit is not affected by applying non-linear transformations to the data, even if these transformations are non-reversible and drastically reduce the information content of the input data. We compare emerging data embeddings produced by supervised and unsupervised training, using MNIST and human EEG recordings during sleep. We find that categories are not only well separated in the final layers of classifiers trained with back-propagation, but to a smaller degree also after unsupervised dimensionality reduction. This suggests that human-defined categories, such as hand-written digits or sleep stages, can indeed be considered as 'natural kinds'.