# Human and AI Perceptual Differences in Image Classification Errors

arXiv: https://arxiv.org/abs/2304.08733 · HTML: https://mlsanity.com/paper/2304.08733/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 1 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 1 of 1)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)

## Built on by (most cited first, 9 of 9)

- [The Importance of Human-Labeled Data in the Era of LLMs](https://mlsanity.com/paper/2306.14910/) (2023)
- [A Survey on Human-AI Collaboration with Large Foundation Models](https://mlsanity.com/paper/2403.04931/) (2024)
- [Learning to Complement and to Defer to Multiple Users](https://mlsanity.com/paper/2407.07003/) (2024)
- [Coverage-Constrained Human-AI Cooperation with Multiple Experts](https://mlsanity.com/paper/2411.11976/) (2024)
- [When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification](https://mlsanity.com/paper/2505.16149/) (2025)
- [No Need for Learning to Defer? A Training Free Deferral Framework to Multiple Experts through Conformal Prediction](https://mlsanity.com/paper/2509.12573/) (2025)
- [Human-AI Collaborative Multi-modal Multi-rater Learning for Endometriosis Diagnosis](https://mlsanity.com/paper/2409.02046/) (2024)
- [Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision](https://mlsanity.com/paper/2604.21909/) (2026)
- [V-REX: Efficient Specialist VLM Training for Veterinary X-Rays](https://mlsanity.com/paper/2608.20069/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) models for computer vision trained with supervised machine learning are assumed to solve classification tasks by imitating human behavior learned from training labels. Most efforts in recent vision research focus on measuring the model task performance using standardized benchmarks such as accuracy. However, limited work has sought to understand the perceptual difference between humans and machines. To fill this gap, this study first analyzes the statistical distributions of mistakes from the two sources and then explores how task difficulty level affects these distributions. We find that even when AI learns an excellent model from the training data, one that outperforms humans in overall accuracy, these AI models have significant and consistent differences from human perception. We demonstrate the importance of studying these differences with a simple human-AI teaming algorithm that outperforms humans alone, AI alone, or AI-AI teaming.