# FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfitting

arXiv: https://arxiv.org/abs/2204.00491 · HTML: https://mlsanity.com/paper/2204.00491/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 51 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 30 of 39)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions](https://mlsanity.com/paper/2003.01826/) (2020)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Understanding and Improving Fast Adversarial Training](https://mlsanity.com/paper/2007.02617/) (2020)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [Geometry-aware Instance-reweighted Adversarial Training](https://mlsanity.com/paper/2010.01736/) (2020)
- [Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach](https://mlsanity.com/paper/1807.04457/) (2018)
- [Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses](https://mlsanity.com/paper/1811.09600/) (2018)
- [Understanding Catastrophic Overfitting in Single-step Adversarial Training](https://mlsanity.com/paper/2010.01799/) (2020)
- [Wavelet Integrated CNNs for Noise-Robust Image Classification](https://mlsanity.com/paper/2005.03337/) (2020)

## Built on by (most cited first, 28 of 28)

- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [CosPGD: an efficient white-box adversarial attack for pixel-wise prediction tasks](https://mlsanity.com/paper/2302.02213/) (2023)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [Frequency Dynamic Convolution for Dense Image Prediction](https://mlsanity.com/paper/2503.18783/) (2025)
- [Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness](https://mlsanity.com/paper/2410.03030/) (2024)
- [When Semantic Segmentation Meets Frequency Aliasing](https://mlsanity.com/paper/2403.09065/) (2024)
- [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025)
- [An Extended Study of Human-like Behavior under Adversarial Training](https://mlsanity.com/paper/2303.12669/) (2023)
- [As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domain](https://mlsanity.com/paper/2307.10001/) (2023)
- [Improving Feature Stability during Upsampling -- Spectral Artifacts and the Importance of Spatial Context](https://mlsanity.com/paper/2311.17524/) (2023)
- [Beware of Aliases -- Signal Preservation is Crucial for Robust Image Restoration](https://mlsanity.com/paper/2406.07435/) (2024)
- [Are Synthetic Corruptions A Reliable Proxy For Real-World Corruptions?](https://mlsanity.com/paper/2505.04835/) (2025)
- [How Do Training Methods Influence the Utilization of Vision Models?](https://mlsanity.com/paper/2410.14470/) (2024)
- [Towards Class-wise Robustness Analysis](https://mlsanity.com/paper/2411.19853/) (2024)
- [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025)
- [Catastrophic overfitting can be induced with discriminative non-robust features](https://mlsanity.com/paper/2206.08242/) (2022)
- [FreqPDE: Rethinking Positional Depth Embedding for Multi-View 3D Object Detection Transformers](https://mlsanity.com/paper/2510.15385/) (2025)
- [Alias-Free ViT: Fractional Shift Invariance via Linear Attention](https://mlsanity.com/paper/2510.22673/) (2025)
- [On the Interplay of Convolutional Padding and Adversarial Robustness](https://mlsanity.com/paper/2308.06612/) (2023)
- [First line of defense: A robust first layer mitigates adversarial attacks](https://mlsanity.com/paper/2408.11680/) (2024)
- [Adversarial Examples in Environment Perception for Automated Driving (Review)](https://mlsanity.com/paper/2504.08414/) (2025)
- [Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling](https://mlsanity.com/paper/2307.09804/) (2023)
- [Fast Adversarial Training against Sparse Attacks Requires Loss Smoothing](https://mlsanity.com/paper/2502.21041/) (2025)
- [DispBench: Benchmarking Disparity Estimation to Synthetic Corruptions](https://mlsanity.com/paper/2505.05091/) (2025)
- [TRIX- Trading Adversarial Fairness via Mixed Adversarial Training](https://mlsanity.com/paper/2507.07768/) (2025)
- [ActiveFreq: Integrating Active Learning and Frequency Domain Analysis for Interactive Segmentation](https://mlsanity.com/paper/2603.11498/) (2026)
- [Deep Psychovisual Image Representations](https://mlsanity.com/paper/2605.29260/) (2026)
- [RL-FAT: Reinforcement Learning for Fair Adversarial Training](https://mlsanity.com/paper/2608.29247/) (2026)

## Abstract (arXiv)

Over the last years, Convolutional Neural Networks (CNNs) have been the dominating neural architecture in a wide range of computer vision tasks. From an image and signal processing point of view, this success might be a bit surprising as the inherent spatial pyramid design of most CNNs is apparently violating basic signal processing laws, i.e. Sampling Theorem in their down-sampling operations. However, since poor sampling appeared not to affect model accuracy, this issue has been broadly neglected until model robustness started to receive more attention. Recent work [17] in the context of adversarial attacks and distribution shifts, showed after all, that there is a strong correlation between the vulnerability of CNNs and aliasing artifacts induced by poor down-sampling operations. This paper builds on these findings and introduces an aliasing free down-sampling operation which can easily be plugged into any CNN architecture: FrequencyLowCut pooling. Our experiments show, that in combination with simple and fast FGSM adversarial training, our hyper-parameter free operator significantly improves model robustness and avoids catastrophic overfitting.