mlsanityData to 7 Oct 2026

Fast Algorithms for Convolutional Neural Networks

arXiv 1509.09308 ↗First posted 30 Sep 2015cs.NE cs.LGcited by 102

Not on the map. Why?

Abstract (arXiv)

Deep convolutional neural networks take GPU days of compute time to train on large data sets. Pedestrian detection for self driving cars requires very low latency. Image recognition for mobile phones is constrained by limited processing resources. The success of convolutional neural networks in these situations is limited by how fast we can compute them. Conventional FFT based convolution is fast for large filters, but state of the art convolutional neural networks use small, 3x3 filters. We introduce a new class of fast algorithms for convolutional neural networks using Winograd's minimal filtering algorithms. The algorithms compute minimal complexity convolution over small tiles, which makes them fast with small filters and small batch sizes. We benchmark a GPU implementation of our algorithm with the VGG network and show state of the art throughput at batch sizes from 1 to 64.

Key facts. Cited by 102 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Sep 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 23; 2023: 18; 2024: 18; 2025: 12; 2026: 6.

102 papers.

Sort:

By area:

  1. 2022A ConvNet for the 2020scited by 3,545
  2. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  3. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  4. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  5. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  6. 2023Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizationscited by 45
  7. 2021RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognitioncited by 40
  8. 2021AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learningcited by 39
  9. 2023Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attentioncited by 38
  10. 2022Re-parameterizing Your Optimizers rather than Architecturescited by 35
  11. 2023Visual Tuningcited by 34
  12. 2021RepMLPNet: Hierarchical Vision MLP with Re-parameterized Localitycited by 32
38 more
  1. 2022Tensor Program Optimization with Probabilistic Programscited by 30
  2. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  3. 2021Dual-side Sparse Tensor Corecited by 27
  4. 2022Fast and Memory-Efficient Network Towards Efficient Image Super-Resolutioncited by 27
  5. 2021A Full-Stack Search Technique for Domain Optimized Deep Learning Acceleratorscited by 19
  6. 2023Layer Collaboration in the Forward-Forward Algorithmcited by 19
  7. 2023Zero-Knowledge Proof-based Verifiable Decentralized Machine Learning in Communication Network: A Comprehensive Surveycited by 18
  8. 20231-Lipschitz Layers Compared: Memory, Speed, and Certifiable Robustnesscited by 15
  9. 2024DiC: Rethinking Conv3x3 Designs in Diffusion Modelscited by 14
  10. 2021Latent Space Explanation by Interventioncited by 13
  11. 2021GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsitycited by 10
  12. 2022FAR: Fourier Aerial Video Recognitioncited by 10
  13. 2021Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniquescited by 9
  14. 2022Winograd Convolution for Deep Neural Networks: Efficient Point Selectioncited by 8
  15. 2022Deformable Butterfly: A Highly Structured and Sparse Linear Transformcited by 8
  16. 2021Learning on Hardware: A Tutorial on Neural Network Accelerators and Co-Processorscited by 6
  17. 2022Winograd Convolution: A Perspective from Fault Tolerancecited by 6
  18. 2022Boosting DNN Cold Inference on Edge Devicescited by 6
  19. 2023PQA: Exploring the Potential of Product Quantization in DNN Hardware Accelerationcited by 6
  20. 2021WinoCNN: Kernel Sharing Winograd Systolic Array for Efficient Convolutional Neural Network Acceleration on FPGAscited by 5
  21. 2022A Functional Information Perspective on Model Interpretationcited by 5
  22. 2021Winograd Algorithm for AdderNetcited by 4
  23. 2021perf4sight: A toolflow to model CNN training performance on Edge GPUscited by 4
  24. 2025Model Compression using Progressive Channel Pruningcited by 4
  25. 2021Learning Versatile Convolution Filters for Efficient Visual Recognitioncited by 3
  26. 2021Speedup deep learning models on GPU by taking advantage of efficient unstructured pruning and bit-width reductioncited by 3
  27. 2022Fast matrix multiplication for binary and ternary CNNs on ARM CPUcited by 3
  28. 2023InferNet: Exploiting Aggregate GPU Profiles as Side-Channel for DNN Architecture Inferencecited by 3
  29. 2023Im2win: An Efficient Convolution Paradigm on GPUcited by 3
  30. 2023Ternary Singular Value Decomposition as a Better Parameterized Form in Linear Mappingcited by 3
  31. 2024CNN Mixture-of-Depthscited by 3
  32. 2022Efficient ML Models for Practical Secure Inferencecited by 2
  33. 2023PAUMER: Patch Pausing Transformer for Semantic Segmentationcited by 2
  34. 2023Fast Inner-Product Algorithms and Architectures for Deep Neural Network Acceleratorscited by 2
  35. 2024Pruner: A Draft-then-Verify Exploration Mechanism to Accelerate Tensor Program Tuningcited by 2
  36. 2024From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networkscited by 2
  37. 2025Floating-Point Neural Network Verification at the Software Levelcited by 2
  38. 2021Accelerating Large Kernel Convolutions with Nested Winograd Transformation.pdfcited by 1

50 most cited of 102 All 102 (JSON)