# Extreme Image Transformations Affect Humans and Machines Differently

arXiv: https://arxiv.org/abs/2212.13967 · HTML: https://mlsanity.com/paper/2212.13967/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 96 references that are in the map.
- First posted on arXiv 30 Nov 2022.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Towards Robust Neural Networks via Random Self-ensemble](https://mlsanity.com/paper/1712.00673/) (2017)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [Robust and Accurate Object Detection via Adversarial Learning](https://mlsanity.com/paper/2103.13886/) (2021)
- [Putting visual object recognition in context](https://mlsanity.com/paper/1911.07349/) (2019)
- [Humans can decipher adversarial images](https://mlsanity.com/paper/1809.04120/) (2018)
- [Towards Robust Adaptive Object Detection under Noisy Annotations](https://mlsanity.com/paper/2204.02620/) (2022)
- [Noise Robust Generative Adversarial Networks](https://mlsanity.com/paper/1911.11776/) (2019)
- [IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos](https://mlsanity.com/paper/2007.06317/) (2020)
- [Tracking Without Re-recognition in Humans and Machines](https://mlsanity.com/paper/2105.13351/) (2021)
- [Neural Population Geometry Reveals the Role of Stochasticity in Robust Perception](https://mlsanity.com/paper/2111.06979/) (2021)
- [Deep-CAPTCHA: a deep learning based CAPTCHA solver for vulnerability assessment](https://mlsanity.com/paper/2006.08296/) (2020)
- [The Challenge of Appearance-Free Object Tracking with Feedforward Neural Networks](https://mlsanity.com/paper/2110.02772/) (2021)
- [Robustness of Humans and Machines on Object Recognition with Extreme Image Transformations](https://mlsanity.com/paper/2205.05167/) (2022)

## Built on by (most cited first, 4 of 4)

- [Towards Experience-Centered AI: A Framework for Integrating Lived Experience in Design and Development](https://mlsanity.com/paper/2508.06849/) (2025)
- [Extreme Image Transformations Facilitate Robust Latent Object Representations](https://mlsanity.com/paper/2310.07725/) (2023)
- [Stretching Beyond the Obvious: A Gradient-Free Framework to Unveil the Hidden Landscape of Visual Invariance](https://mlsanity.com/paper/2506.17040/) (2025)
- [Do Machines Fail Like Humans? A Human-Centred Out-of-Distribution Spectrum for Mapping Error Alignment](https://mlsanity.com/paper/2603.07462/) (2026)

## Abstract (arXiv)

Some recent artificial neural networks (ANNs) claim to model aspects of primate neural and human performance data. Their success in object recognition is, however, dependent on exploiting low-level features for solving visual tasks in a way that humans do not. As a result, out-of-distribution or adversarial input is often challenging for ANNs. Humans instead learn abstract patterns and are mostly unaffected by many extreme image distortions. We introduce a set of novel image transforms inspired by neurophysiological findings and evaluate humans and ANNs on an object recognition task. We show that machines perform better than humans for certain transforms and struggle to perform at par with humans on others that are easy for humans. We quantify the differences in accuracy for humans and machines and find a ranking of difficulty for our transforms for human data. We also suggest how certain characteristics of human visual processing can be adapted to improve the performance of ANNs for our difficult-for-machines transforms.