# Band-limited Training and Inference for Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1911.09287 · HTML: https://mlsanity.com/paper/1911.09287/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2019.

## Built on by (most cited first, 13 of 13)

- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Towards Discriminative Representation Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/2108.03439/) (2021)
- [VUS: Effective and Efficient Accuracy Measures for Time-Series Anomaly Detection](https://mlsanity.com/paper/2502.13318/) (2025)
- [DCT-Former: Efficient Self-Attention with Discrete Cosine Transform](https://mlsanity.com/paper/2203.01178/) (2022)
- [Few-Shot Learning by Integrating Spatial and Frequency Representation](https://mlsanity.com/paper/2105.05348/) (2021)
- [Differentially Private Federated Learning With Time-Adaptive Privacy Spending](https://mlsanity.com/paper/2502.18706/) (2025)
- [Transferability of Graph Neural Networks using Graphon and Sampling Theories](https://mlsanity.com/paper/2307.13206/) (2023)
- [Frequency Regularization: Restricting Information Redundancy of Convolutional Neural Networks](https://mlsanity.com/paper/2304.07973/) (2023)
- [Explicit Use of Fourier Spectrum in Generative Adversarial Networks](https://mlsanity.com/paper/2208.01265/) (2022)
- [MOSAIC: Masked Optimisation with Selective Attention for Image Reconstruction](https://mlsanity.com/paper/2306.00906/) (2023)
- [FreqGRL: Suppressing Low-Frequency Bias and Mining High-Frequency Knowledge for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2511.06648/) (2025)
- [Are Candidate Models Really Needed for Active Learning?](https://mlsanity.com/paper/2605.14689/) (2026)
- [Low-Frequency Shortcuts in Texture-Driven Visual Learning](https://mlsanity.com/paper/2606.03493/) (2026)

## Abstract (arXiv)

The convolutional layers are core building blocks of neural network architectures. In general, a convolutional filter applies to the entire frequency spectrum of the input data. We explore artificially constraining the frequency spectra of these filters and data, called band-limiting, during training. The frequency domain constraints apply to both the feed-forward and back-propagation steps. Experimentally, we observe that Convolutional Neural Networks (CNNs) are resilient to this compression scheme and results suggest that CNNs learn to leverage lower-frequency components. In particular, we found: (1) band-limited training can effectively control the resource usage (GPU and memory); (2) models trained with band-limited layers retain high prediction accuracy; and (3) requires no modification to existing training algorithms or neural network architectures to use unlike other compression schemes.