# A comparison between humans and AI at recognizing objects in unusual poses

arXiv: https://arxiv.org/abs/2402.03973 · HTML: https://mlsanity.com/paper/2402.03973/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 73 references that are in the map.
- First posted on arXiv 6 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Strike (with) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects](https://mlsanity.com/paper/1811.11553/) (2018)
- [Revisiting Weakly Supervised Pre-Training of Visual Perception Models](https://mlsanity.com/paper/2201.08371/) (2022)
- [Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency](https://mlsanity.com/paper/2006.16736/) (2020)
- [Human alignment of neural network representations](https://mlsanity.com/paper/2211.01201/) (2022)
- [Adversarial Attacks Beyond the Image Space](https://mlsanity.com/paper/1711.07183/) (2017)
- [When does dough become a bagel? Analyzing the remaining mistakes on ImageNet](https://mlsanity.com/paper/2205.04596/) (2022)
- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [Recurrence is required to capture the representational dynamics of the human visual system](https://mlsanity.com/paper/1903.05946/) (2019)
- [Recurrent computations for visual pattern completion](https://mlsanity.com/paper/1706.02240/) (2017)
- [Performance-optimized deep neural networks are evolving into worse models of inferotemporal visual cortex](https://mlsanity.com/paper/2306.03779/) (2023)
- [The Robustness Limits of SoTA Vision Models to Natural Variation](https://mlsanity.com/paper/2210.13604/) (2022)
- [Progress and limitations of deep networks to recognize objects in unusual poses](https://mlsanity.com/paper/2207.08034/) (2022)
- [Adversarial alignment: Breaking the trade-off between the strength of an attack and its relevance to human perception](https://mlsanity.com/paper/2306.03229/) (2023)
- [Computing a human-like reaction time metric from stable recurrent vision models](https://mlsanity.com/paper/2306.11582/) (2023)
- [In-distribution adversarial attacks on object recognition models using gradient-free search](https://mlsanity.com/paper/2106.16198/) (2021)

## Built on by (most cited first, 7 of 7)

- [We Can't Understand AI Using our Existing Vocabulary](https://mlsanity.com/paper/2502.07586/) (2025)
- [On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory](https://mlsanity.com/paper/2412.11521/) (2024)
- [Decoupling the components of geometric understanding in Vision Language Models](https://mlsanity.com/paper/2503.03840/) (2025)
- [The Thousand Brains Theory 2.0: An Extension for the Long-Range Connections of the Neocortical Heterarchy](https://mlsanity.com/paper/2507.05888/) (2025)
- [Position: Capability Control Should be a Separate Goal From Alignment](https://mlsanity.com/paper/2602.05164/) (2026)
- [Inverting Data Transformations via Diffusion Sampling](https://mlsanity.com/paper/2602.08267/) (2026)
- [Latent Equivariant Operators for Robust Object Recognition: Promises and Challenges](https://mlsanity.com/paper/2602.18406/) (2026)

## Abstract (arXiv)

Deep learning is closing the gap with human vision on several object recognition benchmarks. Here we investigate this gap for challenging images where objects are seen in unusual poses. We find that humans excel at recognizing objects in such poses. In contrast, state-of-the-art deep networks for vision (EfficientNet, SWAG, ViT, SWIN, BEiT, ConvNext) and state-of-the-art large vision-language models (Claude 3.5, Gemini 1.5, GPT-4) are systematically brittle on unusual poses, with the exception of Gemini showing excellent robustness in that condition. As we limit image exposure time, human performance degrades to the level of deep networks, suggesting that additional mental processes (requiring additional time) are necessary to identify objects in unusual poses. An analysis of error patterns of humans vs. networks reveals that even time-limited humans are dissimilar to feed-forward deep networks. In conclusion, our comparison reveals that humans and deep networks rely on different mechanisms for recognizing objects in unusual poses. Understanding the nature of the mental processes taking place during extra viewing time may be key to reproduce the robustness of human vision in silico.