# Efficient Image Dataset Classification Difficulty Estimation for Predicting Deep-Learning Accuracy

arXiv: https://arxiv.org/abs/1803.09588 · HTML: https://mlsanity.com/paper/1803.09588/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2018.

## Built on by (most cited first, 14 of 14)

- [LOVM: Language-Only Vision Model Selection](https://mlsanity.com/paper/2306.08893/) (2023)
- [Precision-Weighted Federated Learning](https://mlsanity.com/paper/2107.09627/) (2021)
- [A Lightweight Measure of Classification Difficulty from Application Dataset Characteristics](https://mlsanity.com/paper/2404.05981/) (2024)
- [Was that so hard? Estimating human classification difficulty](https://mlsanity.com/paper/2203.11824/) (2022)
- [What can we Learn by Predicting Accuracy?](https://mlsanity.com/paper/2208.01358/) (2022)
- [Predicting the Encoding Error of SIRENs](https://mlsanity.com/paper/2410.21645/) (2024)
- [Few-Class Arena: A Benchmark for Efficient Selection of Vision Models and Dataset Difficulty Measurement](https://mlsanity.com/paper/2411.01099/) (2024)
- [Multimodal LLM With Hierarchical Mixture-of-Experts for VQA on 3D Brain MRI](https://mlsanity.com/paper/2509.25889/) (2025)
- [PEng4NN: An Accurate Performance Estimation Engine for Efficient Automated Neural Network Architecture Search](https://mlsanity.com/paper/2101.04185/) (2021)
- [Dataset-Adaptive Dimensionality Reduction](https://mlsanity.com/paper/2507.11984/) (2025)
- [SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning](https://mlsanity.com/paper/2603.18423/) (2026)
- [Example Perplexity](https://mlsanity.com/paper/2203.08813/) (2022)
- [UnrealNAS: Can We Search Neural Architectures with Unreal Data?](https://mlsanity.com/paper/2205.02162/) (2022)
- [Efficient Neural Network Model Selection for Few-Class Application Datasets](https://mlsanity.com/paper/2606.19712/) (2026)

## Abstract (arXiv)

In the deep-learning community new algorithms are published at an incredible pace. Therefore, solving an image classification problem for new datasets becomes a challenging task, as it requires to re-evaluate published algorithms and their different configurations in order to find a close to optimal classifier. To facilitate this process, before biasing our decision towards a class of neural networks or running an expensive search over the network space, we propose to estimate the classification difficulty of the dataset. Our method computes a single number that characterizes the dataset difficulty 27x faster than training state-of-the-art networks. The proposed method can be used in combination with network topology and hyper-parameter search optimizers to efficiently drive the search towards promising neural-network configurations.