# Truly shift-invariant convolutional neural networks

arXiv: https://arxiv.org/abs/2011.14214 · HTML: https://mlsanity.com/paper/2011.14214/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2020.

## Built on by (most cited first, 30 of 53)

- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Approximation-Generalization Trade-offs under (Approximate) Group Equivariance](https://mlsanity.com/paper/2305.17592/) (2023)
- [FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting](https://mlsanity.com/paper/2204.00491/) (2022)
- [Learning Layer-wise Equivariances Automatically using Gradients](https://mlsanity.com/paper/2310.06131/) (2023)
- [Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks](https://mlsanity.com/paper/2210.08001/) (2022)
- [Shift Invariance Can Reduce Adversarial Robustness](https://mlsanity.com/paper/2103.02695/) (2021)
- [Making Vision Transformers Truly Shift-Equivariant](https://mlsanity.com/paper/2305.16316/) (2023)
- [Group Equivariant Subsampling](https://mlsanity.com/paper/2106.05886/) (2021)
- [Learning Instance-Specific Augmentations by Capturing Local Invariances](https://mlsanity.com/paper/2206.00051/) (2022)
- [Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Space](https://mlsanity.com/paper/2503.09419/) (2025)
- [Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations](https://mlsanity.com/paper/2303.08085/) (2023)
- [Truly shift-equivariant convolutional neural networks with adaptive polyphase upsampling](https://mlsanity.com/paper/2105.04040/) (2021)
- [Robust Perception through Equivariance](https://mlsanity.com/paper/2212.06079/) (2022)
- [Reviving Shift Equivariance in Vision Transformers](https://mlsanity.com/paper/2306.07470/) (2023)
- [Zone Evaluation: Revealing Spatial Bias in Object Detection](https://mlsanity.com/paper/2310.13215/) (2023)
- [Investigating Shift Equivalence of Convolutional Neural Networks in Industrial Defect Segmentation](https://mlsanity.com/paper/2309.16902/) (2023)
- [Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling](https://mlsanity.com/paper/2404.07410/) (2024)
- [SSLChange: A Self-supervised Change Detection Framework Based on Domain Adaptation](https://mlsanity.com/paper/2405.18224/) (2024)
- [Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networks](https://mlsanity.com/paper/2107.00623/) (2021)
- [Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learning](https://mlsanity.com/paper/2502.19921/) (2025)
- [Structure-aware Semantic Discrepancy and Consistency for 3D Medical Image Self-supervised Learning](https://mlsanity.com/paper/2507.02581/) (2025)
- [Channel Selected Stratified Nested Cross Validation for Clinically Relevant EEG Based Parkinsons Disease Detection](https://mlsanity.com/paper/2601.05276/) (2025)
- [Feature-level augmentation to improve robustness of deep neural networks to affine transformations](https://mlsanity.com/paper/2202.05152/) (2022)
- [Using and Abusing Equivariance](https://mlsanity.com/paper/2308.11316/) (2023)
- [Autoequivariant Network Search via Group Decomposition](https://mlsanity.com/paper/2104.04848/) (2021)
- [In-distribution adversarial attacks on object recognition models using gradient-free search](https://mlsanity.com/paper/2106.16198/) (2021)
- [Generalization to translation shifts: a study in architectures and augmentations](https://mlsanity.com/paper/2207.02349/) (2022)
- [Solving Large-scale Spatial Problems with Convolutional Neural Networks](https://mlsanity.com/paper/2306.08191/) (2023)
- [Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex](https://mlsanity.com/paper/2406.16935/) (2024)
- [Group Downsampling with Equivariant Anti-aliasing](https://mlsanity.com/paper/2504.17258/) (2025)

## Abstract (arXiv)

Thanks to the use of convolution and pooling layers, convolutional neural networks were for a long time thought to be shift-invariant. However, recent works have shown that the output of a CNN can change significantly with small shifts in input: a problem caused by the presence of downsampling (stride) layers. The existing solutions rely either on data augmentation or on anti-aliasing, both of which have limitations and neither of which enables perfect shift invariance. Additionally, the gains obtained from these methods do not extend to image patterns not seen during training. To address these challenges, we propose adaptive polyphase sampling (APS), a simple sub-sampling scheme that allows convolutional neural networks to achieve 100% consistency in classification performance under shifts, without any loss in accuracy. With APS, the networks exhibit perfect consistency to shifts even before training, making it the first approach that makes convolutional neural networks truly shift-invariant.