mlsanityData to 7 Oct 2026

FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting

arXiv 2204.00491 ↗First posted 1 Apr 2022cs.CV eess.IVcited by 28

Not on the map. Why?

Abstract (arXiv)

Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks. From an image and signal processing point of view, this success might be a bit surprising as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. Sampling Theorem in their down-sampling operations. However, since poor sampling appeared not to affect model accuracy, this issue has been broadly neglected until model robustness started to receive more attention. Recent work [17] in the context of adversarial attacks and distribution shifts, showed after all, that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by poor down-sampling operations. This paper builds on these findings and introduces an aliasing free down-sampling operation which can easily be plugged into any CNN architecture: FrequencyLowCut pooling. Our experiments show, that in combination with simple and fast FGSM adversarial training, our hyper-parameter free operator significantly improves model robustness and avoids catastrophic overfitting.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 39 of its 51 references that are in the map. First posted on arXiv 1 Apr 2022.

Builds on

39 of 51 references.

References · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  9. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  12. 2016Adversarial Machine Learning at Scalecited by 934
27 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  2. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
  3. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  4. 2020Overfitting in adversarially robust deep learningcited by 504
  5. 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
  6. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  7. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  8. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 315
  9. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 278
  10. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  11. 2021Improving Robustness using Generated Datacited by 233
  12. 2020Understanding and Improving Fast Adversarial Trainingcited by 213
  13. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  14. 2020Geometry-aware Instance-reweighted Adversarial Trainingcited by 198
  15. 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
  16. 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
  17. 2020Understanding Catastrophic Overfitting in Single-step Adversarial Trainingcited by 75
  18. 2020Wavelet Integrated CNNs for Noise-Robust Image Classificationcited by 60
  19. 2020Delving Deeper into Anti-aliasing in ConvNetscited by 60
  20. 2021Relating Adversarially Robust Generalization to Flat Minimacited by 57
  21. 2020Truly shift-invariant convolutional neural networkscited by 53
  22. 2020Spectral Distribution Aware Image Generationcited by 29
  23. 2021WaveCNet: Wavelet Integrated CNNs to Suppress Aliasing Effect for Noise-Robust Image Classificationcited by 29
  24. 2021Is RobustBench/AutoAttack a suitable Benchmark for Adversarial Robustness?cited by 12
  25. 2021Anti-aliasing Deep Image Classifiers using Novel Depth Adaptive Blurring and Activation Functioncited by 11
  26. 2017Wolf in Sheep's Clothing - The Downscaling Attack Against Deep Learning Applicationscited by 1
  27. 2020Downscaling Attack and Defense: Turning What You See Back Into What You Getcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 7; 2025: 10; 2026: 3.

28 papers.

Sort:

  1. 2024Frequency-aware Feature Fusion for Dense Image Predictioncited by 48
  2. 2023CosPGD: an efficient white-box adversarial attack for pixel-wise prediction taskscited by 27
  3. 2022Robust Models are less Over-Confidentcited by 25
  4. 2025Frequency Dynamic Convolution for Dense Image Predictioncited by 19
  5. 2024Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustnesscited by 11
  6. 2024When Semantic Segmentation Meets Frequency Aliasingcited by 10
  7. 2025Frequency-Dynamic Attention Modulation for Dense Predictioncited by 10
  8. 2023An Extended Study of Human-like Behavior under Adversarial Trainingcited by 9
  9. 2023As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domaincited by 8
  10. 2023Improving Feature Stability during Upsampling -- Spectral Artifacts and the Importance of Spatial Contextcited by 8
  11. 2024Beware of Aliases -- Signal Preservation is Crucial for Robust Image Restorationcited by 6
  12. 2025Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?cited by 5
16 more
  1. 2024How Do Training Methods Influence the Utilization of Vision Models?cited by 3
  2. 2024Towards Class-wise Robustness Analysiscited by 3
  3. 2025Spatial Frequency Modulation for Semantic Segmentationcited by 3
  4. 2022Catastrophic overfitting can be induced with discriminative non-robust featurescited by 2
  5. 2025FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformerscited by 2
  6. 2025Alias-Free ViT: Fractional Shift Invariance via Linear Attentioncited by 2
  7. 2023On the Interplay of Convolutional Padding and Adversarial Robustnesscited by 1
  8. 2024First line of defense: A robust first layer mitigates adversarial attackscited by 1
  9. 2025Adversarial Examples in Environment Perception for Automated Driving (Review)cited by 1
  10. 2023Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Poolingcited by 0
  11. 2025Fast Adversarial Training against Sparse Attacks Requires Loss Smoothingcited by 0
  12. 2025DispBench: Benchmarking Disparity Estimation to Synthetic Corruptionscited by 0
  13. 2025TRIX- Trading Adversarial Fairness via Mixed Adversarial Trainingcited by 0
  14. 2026ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentationcited by 0
  15. 2026Deep Psychovisual Image Representationscited by 0
  16. 2026RL-FAT: Reinforcement Learning for Fair Adversarial Trainingcited by 0