# Generalisation in humans and deep neural networks

arXiv: https://arxiv.org/abs/1808.08750 · HTML: https://mlsanity.com/paper/1808.08750/

## Key facts

- Cited by 237 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2018.

## Built on by (most cited first, 30 of 237)

- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [Beyond neural scaling laws: beating power law scaling via data pruning](https://mlsanity.com/paper/2206.14486/) (2022)
- [Test-Time Training with Masked Autoencoders](https://mlsanity.com/paper/2209.07522/) (2022)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Do generative video models understand physical principles?](https://mlsanity.com/paper/2501.09038/) (2025)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Benchmarking Robustness of 3D Object Detection to Common Corruptions in Autonomous Driving](https://mlsanity.com/paper/2303.11040/) (2023)
- [On Interaction Between Augmentations and Corruptions in Natural Corruption Robustness](https://mlsanity.com/paper/2102.11273/) (2021)
- [The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning](https://mlsanity.com/paper/2106.15831/) (2021)
- [Human alignment of neural network representations](https://mlsanity.com/paper/2211.01201/) (2022)
- [On the effectiveness of adversarial training against common corruptions](https://mlsanity.com/paper/2103.02325/) (2021)
- [On the Foundations of Shortcut Learning](https://mlsanity.com/paper/2310.16228/) (2023)
- [Causally motivated Shortcut Removal Using Auxiliary Labels](https://mlsanity.com/paper/2105.06422/) (2021)
- [Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration](https://mlsanity.com/paper/2107.05446/) (2021)
- [Towards Robust Referring Image Segmentation](https://mlsanity.com/paper/2209.09554/) (2022)
- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [ViewFool: Evaluating the Robustness of Visual Recognition to Adversarial Viewpoints](https://mlsanity.com/paper/2210.03895/) (2022)
- [Robust Contrastive Learning Using Negative Samples with Diminished Semantics](https://mlsanity.com/paper/2110.14189/) (2021)
- [When Human Pose Estimation Meets Robustness: Adversarial Algorithms and Benchmarks](https://mlsanity.com/paper/2105.06152/) (2021)
- [Spawrious: A Benchmark for Fine Control of Spurious Correlation Biases](https://mlsanity.com/paper/2303.05470/) (2023)
- [PRIME: A few primitives can boost robustness to common corruptions](https://mlsanity.com/paper/2112.13547/) (2021)
- [Improving neural network representations using human similarity judgments](https://mlsanity.com/paper/2306.04507/) (2023)
- [Physical Design using Differentiable Learned Simulators](https://mlsanity.com/paper/2202.00728/) (2022)
- [ECCV Caption: Correcting False Negatives by Collecting Machine-and-Human-verified Image-Caption Associations for MS-COCO](https://mlsanity.com/paper/2204.03359/) (2022)
- [If your data distribution shifts, use self-learning](https://mlsanity.com/paper/2104.12928/) (2021)
- [Towards Online Domain Adaptive Object Detection](https://mlsanity.com/paper/2204.05289/) (2022)
- [Improving robustness against common corruptions with frequency biased models](https://mlsanity.com/paper/2103.16241/) (2021)
- [Benchmarking the Robustness of Spatial-Temporal Models Against Corruptions](https://mlsanity.com/paper/2110.06513/) (2021)
- [Does enhanced shape bias improve neural network robustness to common corruptions?](https://mlsanity.com/paper/2104.09789/) (2021)
- [Benchmarking Robustness of Adaptation Methods on Pre-trained Vision-Language Models](https://mlsanity.com/paper/2306.02080/) (2023)

## Abstract (arXiv)

We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations. First, using three well known DNNs (ResNet-152, VGG-19, GoogLeNet) we find the human visual system to be more robust to nearly all of the tested image manipulations, and we observe progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker. Secondly, we show that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on, yet they display extremely poor generalisation abilities when tested on other distortion types. For example, training on salt-and-pepper noise does not imply robustness on uniform white noise and vice versa. Thus, changes in the noise distribution between training and testing constitutes a crucial challenge to deep learning vision systems that can be systematically addressed in a lifelong machine learning approach. Our new dataset consisting of 83K carefully measured human psychophysical trials provide a useful reference for lifelong robustness against image degradations set by the human visual system.