mlsanityData to 7 Oct 2026

Impact of Aliasing on Generalization in Deep Convolutional Networks

arXiv 2108.03489 ↗First posted 7 Aug 2021cs.CV cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

We investigate the impact of aliasing on generalization in Deep Convolutional Networks and show that data augmentation schemes alone are unable to prevent it due to structural limitations in widely used architectures. Drawing insights from frequency analysis theory, we take a closer look at ResNet and EfficientNet architectures and review the trade-off between aliasing and information loss in each of their major components. We show how to mitigate aliasing by inserting non-trainable low-pass filters at key locations, particularly where networks lack the capacity to learn them. These simple architectural changes lead to substantial improvements in generalization on i.i.d. and even more on out-of-distribution conditions, such as image classification under natural corruptions on ImageNet-C [11] and few-shot learning on Meta-Dataset [26]. State-of-the art results are achieved on both datasets without introducing additional trainable parameters and using the default hyper-parameters of open source codebases.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 22 of its 32 references that are in the map. First posted on arXiv 7 Aug 2021.

Builds on

22 of 32 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  8. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
  9. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  10. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  11. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  12. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
10 more
  1. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  2. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  3. 2019A Fourier Perspective on Model Robustness in Computer Visioncited by 288
  4. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  5. 2019Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processescited by 131
  6. 2020Smooth Adversarial Trainingcited by 89
  7. 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
  8. 2020Delving Deeper into Anti-aliasing in ConvNetscited by 60
  9. 2020A Corrective View of Neural Networks: Representation, Memorization and Learningcited by 13
  10. 2020Selecting Relevant Features from a Multi-domain Representation for Few-shot Classificationcited by 11

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 6; 2023: 10; 2024: 2; 2025: 3.

25 papers.

Sort:

By area:

  1. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  2. 2021FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizescited by 66
  3. 2023Towards Stability of Autoregressive Neural Operatorscited by 54
  4. 2022The Lie Derivative for Measuring Learned Equivariancecited by 51
  5. 2022Shift-tolerant Perceptual Similarity Metriccited by 44
  6. 2021ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenationcited by 41
  7. 2024Frequency-Adaptive Dilated Convolution for Semantic Segmentationcited by 34
  8. 2023Temporally Consistent Online Depth Estimation Using Point-Based Fusioncited by 12
  9. 2021Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Functioncited by 11
  10. 2022How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?cited by 11
  11. 2021Exploration into Translation-Equivariant Image Quantizationcited by 10
  12. 2024HFI: A unified framework for training-free detection and implicit watermarking of latent diffusion model generated imagescited by 9
13 more
  1. 2022Dynamic Test-Time Augmentation via Differentiable Functionscited by 4
  2. 2023MULLER: Multilayer Laplacian Resizer for Visioncited by 4
  3. 2023DNArch: Learning Convolutional Neural Architectures by Backpropagationcited by 3
  4. 2025Group Downsampling with Equivariant Anti-aliasingcited by 3
  5. 2023The Importance of Anti-Aliasing in Tiny Object Detectioncited by 2
  6. 2022Aliasing is a Driver of Adversarial Attackscited by 1
  7. 2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
  8. 2025Gradient-Free Adversarial Purification with Diffusion Modelscited by 1
  9. 2025WD-DETR: Wavelet Denoising-Enhanced Real-Time Object Detection Transformer for Robot Perception with Event Camerascited by 1
  10. 2022Studying inductive biases in image classification taskcited by 0
  11. 2023Anti-aliasing Predictive Coding Network for Future Video Frame Predictioncited by 0
  12. 2023Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Poolingcited by 0
  13. 2023A New Multimodal Medical Image Fusion based on Laplacian Autoencoder with Channel Attentioncited by 0