mlsanityData to 13 Oct 2026

Fast Training of Convolutional Networks through FFTs

arXiv 1312.5851 ↗First posted 20 Dec 2013cs.CV cs.LG cs.NEcited by 89

Not on the map. Why?

Abstract (arXiv)

Convolutional networks are one of the most widely employed architectures in computer vision and machine learning. In order to leverage their ability to learn complex functions, large amounts of data are required for training. Training a large convolutional network to produce state-of-the-art results can take weeks, even when using modern GPUs. Producing labels using a trained network can also be costly when dealing with web-scale datasets. In this work, we present a simple algorithm which accelerates training and inference by a significant factor, and can yield improvements of over an order of magnitude compared to existing state-of-the-art implementations. This is done by computing convolutions as pointwise products in the Fourier domain while reusing the same transformed feature map many times. The algorithm is implemented on a GPU architecture and addresses a number of related challenges.

Key facts. Cited by 89 papers in the map (data through 13 Oct 2026). First posted on arXiv 20 Dec 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 26; 2022: 21; 2023: 8; 2024: 13; 2025: 14; 2026: 7.

89 papers.

Sort:

By area:

  1. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,206
  2. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  3. 2021FNet: Mixing Tokens with Fourier Transformscited by 346
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  5. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  6. 2022Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNscited by 155
  7. 2022ETSformer: Exponential Smoothing Transformers for Time-series Forecastingcited by 146
  8. 2023A Mathematical Guide to Operator Learningcited by 75
  9. 2022Deep Frequency Filtering for Domain Generalizationcited by 43
  10. 2021RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognitioncited by 40
  11. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  12. 2024DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Spacecited by 33
38 more
  1. 2021RepMLPNet: Hierarchical Vision MLP with Re-parameterized Localitycited by 32
  2. 2024Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformerscited by 32
  3. 2023Frequency Domain-based Dataset Distillationcited by 28
  4. 2022projUNN: efficient method for training deep networks with unitary matricescited by 27
  5. 2022Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flowcited by 20
  6. 2022Transform Once: Efficient Operator Learning in Frequency Domaincited by 20
  7. 2025Frequency Dynamic Convolution for Dense Image Predictioncited by 19
  8. 2023Truly Scale-Equivariant Deep Nets with Fourier Layerscited by 16
  9. 2022Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruningcited by 13
  10. 2021Spectral Leakage and Rethinking the Kernel Size in CNNscited by 11
  11. 2021Accelerating Neural ODEs Using Model Order Reductioncited by 9
  12. 2021Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNNcited by 9
  13. 2021Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniquescited by 9
  14. 2022Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approachcited by 9
  15. 2023As large as it gets: Learning infinitely large Filters via Neural Implicit Functions in the Fourier Domaincited by 8
  16. 2022k-strip: A novel segmentation algorithm in k-space for the application of skull strippingcited by 7
  17. 2024Adaptive Sampling of k-Space in Magnetic Resonance for Rapid Pathology Predictioncited by 7
  18. 2024Distributed-Order Fractional Graph Operating Networkcited by 7
  19. 2021FAT: Learning Low-Bitwidth Parametric Representation via Frequency-Aware Transformationcited by 6
  20. 2021Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networkscited by 6
  21. 2021Unsupervised Image Decomposition with Phase-Correlation Networkscited by 6
  22. 2022Fast-FNet: Accelerating Transformer Encoder Models via Efficient Fourier Layerscited by 5
  23. 2022FCT-GAN: Enhancing Table Synthesis via Fourier Transformcited by 5
  24. 2021Winograd Algorithm for AdderNetcited by 4
  25. 2021perf4sight: A toolflow to model CNN training performance on Edge GPUscited by 4
  26. 2025Model Compression using Progressive Channel Pruningcited by 4
  27. 2021FourierNets enable the design of highly non-local optical encoders for computational imagingcited by 3
  28. 2022Can we integrate spatial verification methods into neural-network loss functions for atmospheric science?cited by 3
  29. 2022Neural Fourier Shift for Binaural Speech Renderingcited by 3
  30. 2022PIP: Positional-encoding Image Priorcited by 3
  31. 2023On the Feasibility of Machine Learning Augmented Magnetic Resonance for Point-of-Care Identification of Diseasecited by 3
  32. 2025Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagationcited by 3
  33. 2024From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networkscited by 2
  34. 2024S3Attention: Improving Long Sequence Attention with Smoothed Skeleton Sketchingcited by 2
  35. 2025Hardware-Efficient Photonic Tensor Core: Accelerating Deep Neural Networks with Structured Compressioncited by 2
  36. 2025Reverse Convolution and Its Applications to Image Restorationcited by 2
  37. 2026Frame2Freq: Spectral Adapters for Fine-Grained Video Understandingcited by 2
  38. 2021cuConv: A CUDA Implementation of Convolution for CNN Inferencecited by 1

50 most cited of 89 All 89 (JSON)