# Convolutional Neural Networks Are Not Invariant to Translation, but They Can Learn to Be

arXiv: https://arxiv.org/abs/2110.05861 · HTML: https://mlsanity.com/paper/2110.05861/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [Meta-Learning Representations for Continual Learning](https://mlsanity.com/paper/1905.12588/) (2019)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Learning to Continually Learn](https://mlsanity.com/paper/2002.09571/) (2020)
- [Multi-scale Orderless Pooling of Deep Convolutional Activation Features](https://mlsanity.com/paper/1403.1840/) (2014)
- [Quantifying Translation-Invariance in Convolutional Neural Networks](https://mlsanity.com/paper/1801.01450/) (2017)
- [Learning rotation invariant convolutional filters for texture classification](https://mlsanity.com/paper/1604.06720/) (2016)
- [A neural network walks into a lab: towards using deep nets as models for human behavior](https://mlsanity.com/paper/2005.02181/) (2020)
- [Five Points to Check when Comparing Visual Perception in Humans and Machines](https://mlsanity.com/paper/2004.09406/) (2020)
- [Minimal Images in Deep Neural Networks: Fragile Object Recognition in Natural Images](https://mlsanity.com/paper/1902.03227/) (2019)
- [Active Long Term Memory Networks](https://mlsanity.com/paper/1606.02355/) (2016)
- [Deep Learning for Target Classification from SAR Imagery: Data Augmentation and Translation Invariance](https://mlsanity.com/paper/1708.07920/) (2017)
- [A case for robust translation tolerance in humans and CNNs. A commentary on Han et al](https://mlsanity.com/paper/2012.05950/) (2020)
- [Representation of White- and Black-Box Adversarial Examples in Deep Neural Networks and Humans: A Functional Magnetic Resonance Imaging Study](https://mlsanity.com/paper/1905.02422/) (2019)

## Built on by (most cited first, 19 of 19)

- [Catch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder](https://mlsanity.com/paper/2305.02541/) (2023)
- [The Robustness Limits of SoTA Vision Models to Natural Variation](https://mlsanity.com/paper/2210.13604/) (2022)
- [Mixed Evidence for Gestalt Grouping in Deep Neural Networks](https://mlsanity.com/paper/2203.07302/) (2022)
- [Stochastic Latent Transformer: Efficient Modelling of Stochastically Forced Zonal Jets](https://mlsanity.com/paper/2310.16741/) (2023)
- [MindSet: Vision. A toolbox for testing DNNs on key psychological experiments](https://mlsanity.com/paper/2404.05290/) (2024)
- [PRUE: A Practical Recipe for Field Boundary Segmentation at Scale](https://mlsanity.com/paper/2603.27101/) (2026)
- [Geometric Data Augmentations to Mitigate Distribution Shifts in Pollen Classification from Microscopic Images](https://mlsanity.com/paper/2311.11029/) (2023)
- [Corner Cases: How Size and Position of Objects Challenge ImageNet-Trained Models](https://mlsanity.com/paper/2505.03569/) (2025)
- [Symmetries in PAC-Bayesian Learning](https://mlsanity.com/paper/2510.17303/) (2025)
- [From CNNs to Shift-Invariant Twin Models Based on Complex Wavelets](https://mlsanity.com/paper/2212.00394/) (2022)
- [Subspace-Configurable Networks](https://mlsanity.com/paper/2305.13536/) (2023)
- [Scale-Translation Equivariant Network for Oceanic Internal Solitary Wave Localization](https://mlsanity.com/paper/2406.13060/) (2024)
- [Knowledge distillation to effectively attain both region-of-interest and global semantics from an image where multiple objects appear](https://mlsanity.com/paper/2407.08257/) (2024)
- [Learning to Adapt to Position Bias in Vision Transformer Classifiers](https://mlsanity.com/paper/2505.13137/) (2025)
- [What Do You Need for Compositional Generalization in Diffusion Planning?](https://mlsanity.com/paper/2505.18083/) (2025)
- [Neurosymbolic Deep Learning Semantics](https://mlsanity.com/paper/2511.02825/) (2025)
- [Probing the Probes: Methods and Metrics for Concept Alignment](https://mlsanity.com/paper/2511.04312/) (2025)
- [Parameter-Efficient Architectural Modifications for Translation-Invariant CNNs](https://mlsanity.com/paper/2604.27870/) (2026)
- [Shape-Based Inductive Bias for Glioma Grading from Tumor Contours](https://mlsanity.com/paper/2607.26090/) (2026)

## Abstract (arXiv)

When seeing a new object, humans can immediately recognize it across different retinal locations: the internal object representation is invariant to translation. It is commonly believed that Convolutional Neural Networks (CNNs) are architecturally invariant to translation thanks to the convolution and/or pooling operations they are endowed with. In fact, several studies have found that these networks systematically fail to recognise new objects on untrained locations. In this work, we test a wide variety of CNNs architectures showing how, apart from DenseNet-121, none of the models tested was architecturally invariant to translation. Nevertheless, all of them could learn to be invariant to translation. We show how this can be achieved by pretraining on ImageNet, and it is sometimes possible with much simpler data sets when all the items are fully translated across the input canvas. At the same time, this invariance can be disrupted by further training due to catastrophic forgetting/interference. These experiments show how pretraining a network on an environment with the right `latent' characteristics (a more naturalistic environment) can result in the network learning deep perceptual rules which would dramatically improve subsequent generalization.