mlsanityData to 14 Oct 2026

Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling

arXiv 2404.07410 ↗First posted 11 Apr 2024cs.CV cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

Downsampling operators break the shift invariance of convolutional neural networks (CNNs) and this affects the robustness of features learned by CNNs when dealing with even small pixel-level shift. Through a large-scale correlation analysis framework, we study shift invariance of CNNs by inspecting existing downsampling operators in terms of their maximum-sampling bias (MSB), and find that MSB is negatively correlated with shift invariance. Based on this crucial insight, we propose a learnable pooling operator called Translation Invariant Polyphase Sampling (TIPS) and two regularizations on the intermediate feature maps of TIPS to reduce MSB and learn translation-invariant representations. TIPS can be integrated into any CNN and can be trained end-to-end with marginal computational overhead. Our experiments demonstrate that TIPS results in consistent performance gains in terms of accuracy, shift consistency, and shift fidelity on multiple benchmarks for image classification and semantic segmentation compared to previous methods and also leads to improvements in adversarial and distributional robustness. TIPS results in the lowest MSB compared to all previous methods, thus explaining our strong empirical results.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 52 of its 74 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

52 of 74 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  9. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  11. 2016Densely Connected Convolutional Networkscited by 6,720
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
38 more
  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2016Layer Normalizationcited by 5,669
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  10. 2018Group Normalizationcited by 1,524
  11. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,188
  12. 2016Group Equivariant Convolutional Networkscited by 1,165
  13. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
  14. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  15. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 589
  16. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  17. 2017Dilated Residual Networkscited by 294
  18. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  19. 2020Robust and Generalizable Visual Representation Learning via Random Convolutionscited by 142
  20. 2020Delving Deeper into Anti-aliasing in ConvNetscited by 60
  21. 2019Effective Rotation-invariant Point CNN with Spherical Harmonics kernelscited by 57
  22. 2020Truly shift-invariant convolutional neural networkscited by 53
  23. 2019GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNscited by 47
  24. 2022ID and OOD Performance Are Sometimes Inversely Correlated on Real-world Datasetscited by 44
  25. 2023DilateFormer: Multi-Scale Dilated Transformer for Visual Recognitioncited by 34
  26. 2019LIP: Local Importance-based Poolingcited by 30
  27. 2020Attribute-Guided Adversarial Training for Robustness to Natural Perturbationscited by 23
  28. 2022Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustnesscited by 23
  29. 2023The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networkscited by 23
  30. 2023Adversarial Bayesian Augmentation for Single-Source Domain Generalizationcited by 22
  31. 2022Explicit Tradeoffs between Adversarial and Natural Distributional Robustnesscited by 20
  32. 2022Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networkscited by 18
  33. 2021Shift Invariance Can Reduce Adversarial Robustnesscited by 16
  34. 2022Improving Diversity with Adversarially Learned Transformations for Domain Generalizationcited by 16
  35. 2023Making Vision Transformers Truly Shift-Equivariantcited by 16
  36. 2020Shift Equivariance in Object Detectioncited by 13
  37. 2023Reviving Shift Equivariance in Vision Transformerscited by 9
  38. 2023Improving Equivariance in State-of-the-Art Supervised Depth and Normal Predictorscited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Spacecited by 14
  2. 2026To Erase, or Not to Erase: Robust Training-Free Concept Erasure with Preservation aware Adaptive Ranked Subspace Expansioncited by 3
  3. 2025Translation-Equivariance of Normalization Layers and Aliasing in Convolutional Neural Networkscited by 1
  4. 2026OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolutioncited by 1
  5. 2026BEFORE THE FLIP: Measuring Hidden Score Shifts In Quantized Vision Language Models Before The Answer Changes for Visual Question Answeringcited by 1
  6. 2026New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Modelscited by 0