# Stride and Translation Invariance in CNNs

arXiv: https://arxiv.org/abs/2103.10097 · HTML: https://mlsanity.com/paper/2103.10097/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 18 Mar 2021.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Understanding image representations by measuring their equivariance and equivalence](https://mlsanity.com/paper/1411.5908/) (2014)
- [Why do deep convolutional networks generalize so poorly to small image transformations?](https://mlsanity.com/paper/1805.12177/) (2018)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Convolutional Neural Networks at Constrained Time Cost](https://mlsanity.com/paper/1412.1710/) (2014)
- [Quantifying Translation-Invariance in Convolutional Neural Networks](https://mlsanity.com/paper/1801.01450/) (2017)

## Built on by (most cited first, 7 of 7)

- [How Tempering Fixes Data Augmentation in Bayesian Neural Networks](https://mlsanity.com/paper/2205.13900/) (2022)
- [Impoola: The Power of Average Pooling for Image-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2503.05546/) (2025)
- [Just a Matter of Scale? Reevaluating Scale Equivariance in Convolutional Neural Networks](https://mlsanity.com/paper/2211.10288/) (2022)
- [Web Photo Source Identification based on Neural Enhanced Camera Fingerprint](https://mlsanity.com/paper/2302.09228/) (2023)
- [Deep learning complete intersection Calabi-Yau manifolds](https://mlsanity.com/paper/2311.11847/) (2023)
- [Learning Audio Representations with MLPs](https://mlsanity.com/paper/2203.08490/) (2022)
- [A deep learning approach for detection and localization of leaf anomalies](https://mlsanity.com/paper/2210.03558/) (2022)

## Abstract (arXiv)

Convolutional Neural Networks have become the standard for image classification tasks, however, these architectures are not invariant to translations of the input image. This lack of invariance is attributed to the use of stride which ignores the sampling theorem, and fully connected layers which lack spatial reasoning. We show that stride can greatly benefit translation invariance given that it is combined with sufficient similarity between neighbouring pixels, a characteristic which we refer to as local homogeneity. We also observe that this characteristic is dataset-specific and dictates the relationship between pooling kernel size and stride required for translation invariance. Furthermore we find that a trade-off exists between generalization and translation invariance in the case of pooling kernel size, as larger kernel sizes lead to better invariance but poorer generalization. Finally we explore the efficacy of other solutions proposed, namely global average pooling, anti-aliasing, and data augmentation, both empirically and through the lens of local homogeneity.