# Spectral Leakage and Rethinking the Kernel Size in CNNs

arXiv: https://arxiv.org/abs/2101.10143 · HTML: https://mlsanity.com/paper/2101.10143/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 66 references that are in the map.
- First posted on arXiv 25 Jan 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [On the Relationship between Self-Attention and Convolutional Layers](https://mlsanity.com/paper/1911.03584/) (2019)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Orthogonal Convolutional Neural Networks](https://mlsanity.com/paper/1911.12207/) (2019)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [Superposition of many models into one](https://mlsanity.com/paper/1902.05522/) (2019)
- [Gabor Convolutional Networks](https://mlsanity.com/paper/1705.01450/) (2017)
- [Deep Roto-Translation Scattering for Object Classification](https://mlsanity.com/paper/1412.8659/) (2014)

## Built on by (most cited first, 11 of 11)

- [Zero-Shot Day-Night Domain Adaptation with a Physics Prior](https://mlsanity.com/paper/2108.05137/) (2021)
- [As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domain](https://mlsanity.com/paper/2307.10001/) (2023)
- [Frequency learning for structured CNN filters with Gaussian fractional derivatives](https://mlsanity.com/paper/2111.06660/) (2021)
- [SMPConv: Self-moving Point Representations for Continuous Convolution](https://mlsanity.com/paper/2304.02330/) (2023)
- [PSO-Convolutional Neural Networks with Heterogeneous Learning Rate](https://mlsanity.com/paper/2205.10456/) (2022)
- [On the Interplay of Convolutional Padding and Adversarial Robustness](https://mlsanity.com/paper/2308.06612/) (2023)
- [Analysis of Long Range Dependency Understanding in State Space Models](https://mlsanity.com/paper/2601.13048/) (2026)
- [Fix your downsampling ASAP! Be natively more robust via Aliasing and Spectral Artifact free Pooling](https://mlsanity.com/paper/2307.09804/) (2023)
- [ISAC: An Invertible and Stable Auditory Filter Bank with Customizable Kernels for ML Integration](https://mlsanity.com/paper/2505.07709/) (2025)
- [Finding Optimal Kernel Size and Dimension in Convolutional Neural Networks An Architecture Optimization Approach](https://mlsanity.com/paper/2506.14846/) (2025)
- [LAID: Lightweight AI-Generated Image Detection in Spatial and Spectral Domains](https://mlsanity.com/paper/2507.05162/) (2025)

## Abstract (arXiv)

Convolutional layers in CNNs implement linear filters which decompose the input into different frequency bands. However, most modern architectures neglect standard principles of filter design when optimizing their model choices regarding the size and shape of the convolutional kernel. In this work, we consider the well-known problem of spectral leakage caused by windowing artifacts in filtering operations in the context of CNNs. We show that the small size of CNN kernels make them susceptible to spectral leakage, which may induce performance-degrading artifacts. To address this issue, we propose the use of larger kernel sizes along with the Hamming window function to alleviate leakage in CNN architectures. We demonstrate improved classification accuracy on multiple benchmark datasets including Fashion-MNIST, CIFAR-10, CIFAR-100 and ImageNet with the simple use of a standard window function in convolutional layers. Finally, we show that CNNs employing the Hamming window display increased robustness against various adversarial attacks.