mlsanityData to 7 Oct 2026

Truly shift-invariant convolutional neural networks

arXiv 2011.14214 ↗First posted 28 Nov 2020cs.CV cs.AI cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Thanks to the use of convolution and pooling layers, convolutional neural networks were for a long time thought to be shift-invariant. However, recent works have shown that the output of a CNN can change significantly with small shifts in input: a problem caused by the presence of downsampling (stride) layers. The existing solutions rely either on data augmentation or on anti-aliasing, both of which have limitations and neither of which enables perfect shift invariance. Additionally, the gains obtained from these methods do not extend to image patterns not seen during training. To address these challenges, we propose adaptive polyphase sampling (APS), a simple sub-sampling scheme that allows convolutional neural networks to achieve 100% consistency in classification performance under shifts, without any loss in accuracy. With APS, the networks exhibit perfect consistency to shifts even before training, making it the first approach that makes convolutional neural networks truly shift-invariant.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 14; 2023: 10; 2024: 6; 2025: 12; 2026: 3.

53 papers.

Sort:

By area:

  1. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  2. 2023Approximation-Generalization Trade-offs under (Approximate) Group Equivariancecited by 45
  3. 2022FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfittingcited by 28
  4. 2023Learning Layer-wise Equivariances Automatically using Gradientscited by 23
  5. 2022Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networkscited by 18
  6. 2021Shift Invariance Can Reduce Adversarial Robustnesscited by 16
  7. 2023Making Vision Transformers Truly Shift-Equivariantcited by 16
  8. 2021Group Equivariant Subsamplingcited by 15
  9. 2022Learning Instance-Specific Augmentations by Capturing Local Invariancescited by 15
  10. 2025Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Spacecited by 14
  11. 2023Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activationscited by 11
  12. 2021Truly shift-equivariant convolutional neural networks with adaptive polyphase upsamplingcited by 10
38 more
  1. 2022Robust Perception through Equivariancecited by 9
  2. 2023Reviving Shift Equivariance in Vision Transformerscited by 9
  3. 2023Zone Evaluation: Revealing Spatial Bias in Object Detectioncited by 7
  4. 2023Investigating Shift Equivalence of Convolutional Neural Networks in Industrial Defect Segmentationcited by 6
  5. 2024Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Samplingcited by 6
  6. 2024SSLChange: A Self-supervised Change Detection Framework Based on Domain Adaptationcited by 6
  7. 2021Improving Sound Event Classification by Increasing Shift Invariance in Convolutional Neural Networkscited by 5
  8. 2025Shifting the Paradigm: A Diffeomorphism Between Time Series Data Manifolds for Achieving Shift-Invariancy in Deep Learningcited by 5
  9. 2022Feature-level augmentation to improve robustness of deep neural networks to affine transformationscited by 4
  10. 2023Using and Abusing Equivariancecited by 4
  11. 2025Structure-aware Semantic Discrepancy and Consistency for 3D Medical Image Self-supervised Learningcited by 4
  12. 2025Channel Selected Stratified Nested Cross Validation for Clinically Relevant EEG Based Parkinsons Disease Detectioncited by 4
  13. 2021Autoequivariant Network Search via Group Decompositioncited by 3
  14. 2021In-distribution adversarial attacks on object recognition models using gradient-free searchcited by 3
  15. 2022Generalization to translation shifts: a study in architectures and augmentationscited by 3
  16. 2023Solving Large-scale Spatial Problems with Convolutional Neural Networkscited by 3
  17. 2024Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortexcited by 3
  18. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
  19. 2021ASAT: Adaptively Scaled Adversarial Training in Time Seriescited by 2
  20. 2022Improving generalization by mimicking the human visual dietcited by 2
  21. 2022On the Shift Invariance of Max Pooling Feature Maps in Convolutional Neural Networkscited by 2
  22. 2024LoFi: Neural Local Fields for Scalable Image Reconstructioncited by 2
  23. 2025Alias-Free ViT: Fractional Shift Invariance via Linear Attentioncited by 2
  24. 2025Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machinescited by 2
  25. 2022Logarithmic Morphological Neural Nets robust to lighting variationscited by 1
  26. 2022Convolutional layers are equivariant to discrete shifts but not continuous translationscited by 1
  27. 2022Aliasing is a Driver of Adversarial Attackscited by 1
  28. 2023Evaluating Machine Learning Models with NERO: Non-Equivariance Revealed on Orbitscited by 1
  29. 2024Don't Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNscited by 1
  30. 2025Translation-Equivariance of Normalization Layers and Aliasing in Convolutional Neural Networkscited by 1
  31. 2025Equivariant Splitting: Self-supervised learning from incomplete datacited by 1
  32. 2022Shift Variance in Scene Text Detectioncited by 0
  33. 2022Deep Invertible Approximation of Topologically Rich Maps between Manifoldscited by 0
  34. 2022From CNNs to Shift-Invariant Twin Models Based on Complex Waveletscited by 0
  35. 2024Adaptive Sampling for Continuous Group Equivariant Neural Networkscited by 0
  36. 2025What Do You Need for Compositional Generalization in Diffusion Planning?cited by 0
  37. 2025Towards Foundation Models for Cryo-ET Subtomogram Analysiscited by 0
  38. 2025Shift-Equivariant Complex-Valued Convolutional Neural Networkscited by 0

50 most cited of 53 All 53 (JSON)