# Spectral Representations for Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1506.03767 · HTML: https://mlsanity.com/paper/1506.03767/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2015.

## Built on by (most cited first, 30 of 81)

- [Intriguing Findings of Frequency Selection for Image Deblurring](https://mlsanity.com/paper/2111.11745/) (2021)
- [Towards Stability of Autoregressive Neural Operators](https://mlsanity.com/paper/2306.10619/) (2023)
- [FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.12963/) (2024)
- [A Survey on Deep Learning based Time Series Analysis with Frequency Transformation](https://mlsanity.com/paper/2302.02173/) (2023)
- [Deep Fourier Up-Sampling](https://mlsanity.com/paper/2210.05171/) (2022)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Learning strides in convolutional neural networks](https://mlsanity.com/paper/2202.01653/) (2022)
- [Towards General-Purpose Representation Learning of Polygonal Geometries](https://mlsanity.com/paper/2209.15458/) (2022)
- [Transform Once: Efficient Operator Learning in Frequency Domain](https://mlsanity.com/paper/2211.14453/) (2022)
- [LoFormer: Local Frequency Transformer for Image Deblurring](https://mlsanity.com/paper/2407.16993/) (2024)
- [Hyperparameter Transfer Learning with Adaptive Complexity](https://mlsanity.com/paper/2102.12810/) (2021)
- [Image Completion with Heterogeneously Filtered Spectral Hints](https://mlsanity.com/paper/2211.03700/) (2022)
- [OPE-SR: Orthogonal Position Encoding for Designing a Parameter-free Upsampling Module in Arbitrary-scale Image Super-Resolution](https://mlsanity.com/paper/2303.01091/) (2023)
- [Binary Complex Neural Network Acceleration on FPGA](https://mlsanity.com/paper/2108.04811/) (2021)
- [SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation](https://mlsanity.com/paper/2308.11568/) (2023)
- [RCA-IUnet: A residual cross-spatial attention guided inception U-Net model for tumor segmentation in breast ultrasound imaging](https://mlsanity.com/paper/2108.02508/) (2021)
- [LiftPool: Bidirectional ConvNet Pooling](https://mlsanity.com/paper/2104.00996/) (2021)
- [Simple Pooling Front-ends For Efficient Audio Classification](https://mlsanity.com/paper/2210.00943/) (2022)
- [Towards Quantum Graph Neural Networks: An Ego-Graph Learning Approach](https://mlsanity.com/paper/2201.05158/) (2022)
- [k-strip: A novel segmentation algorithm in k-space for the application of skull stripping](https://mlsanity.com/paper/2205.09706/) (2022)
- [FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformation](https://mlsanity.com/paper/2102.07444/) (2021)
- [Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks](https://mlsanity.com/paper/2102.13323/) (2021)
- [EZCrop: Energy-Zoned Channels for Robust Output Pruning](https://mlsanity.com/paper/2105.03679/) (2021)
- [Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layers](https://mlsanity.com/paper/2209.12816/) (2022)
- [Phase-driven Domain Generalizable Learning for Nonstationary Time Series](https://mlsanity.com/paper/2402.05960/) (2024)
- [MGT-Prism: Enhancing Domain Generalization for Machine-Generated Text Detection via Spectral Alignment](https://mlsanity.com/paper/2508.13768/) (2025)
- [LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering](https://mlsanity.com/paper/2111.14547/) (2021)
- [Spectral Cross-Domain Neural Network with Soft-adaptive Threshold Spectral Enhancement](https://mlsanity.com/paper/2301.10171/) (2023)
- [DNArch: Learning Convolutional Neural Architectures by Backpropagation](https://mlsanity.com/paper/2302.05400/) (2023)
- [FourierNets enable the design of highly non-local optical encoders for computational imaging](https://mlsanity.com/paper/2104.10611/) (2021)

## Abstract (arXiv)

Discrete Fourier transforms provide a significant speedup in the computation of convolutions in deep learning. In this work, we demonstrate that, beyond its advantages for efficient computation, the spectral domain also provides a powerful representation in which to model and train convolutional neural networks (CNNs). We employ spectral representations to introduce a number of innovations to CNN design. First, we propose spectral pooling, which performs dimensionality reduction by truncating the representation in the frequency domain. This approach preserves considerably more information per parameter than other pooling strategies and enables flexibility in the choice of pooling output dimensionality. This representation also enables a new form of stochastic regularization by randomized modification of resolution. We show that these methods achieve competitive results on classification and approximation tasks, without using any dropout or max-pooling. Finally, we demonstrate the effectiveness of complex-coefficient spectral parameterization of convolutional filters. While this leaves the underlying model unchanged, it results in a representation that greatly facilitates optimization. We observe on a variety of popular CNN configurations that this leads to significantly faster convergence during training.