mlsanityData to 7 Oct 2026

Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation

arXiv 1404.0736 ↗First posted 2 Apr 2014cs.CV cs.LGcited by 304

Not on the map. Why?

Abstract (arXiv)

We present techniques for speeding up the test-time evaluation of large convolutional networks, designed for object recognition tasks. These models deliver impressive accuracy but each image evaluation requires millions of floating point operations, making their deployment on smartphones and Internet-scale clusters problematic. The computation is dominated by the convolution operations in the lower layers of the model. We exploit the linear structure present within the convolutional filters to derive approximations that significantly reduce the required computation. Using large state-of-the-art models, we demonstrate we demonstrate speedups of convolutional layers on both CPU and GPU by a factor of 2x, while keeping the accuracy within 1% of the original model.

Key facts. Cited by 304 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 69; 2022: 62; 2023: 46; 2024: 39; 2025: 44; 2026: 44.

304 papers.

Sort:

By area:

  1. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  2. 2021NeRV: Neural Representations for Videoscited by 255
  3. 2021Concealed Object Detectioncited by 212
  4. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  5. 2022Language model compression with weighted low-rank factorizationcited by 192
  6. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  7. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 156
  8. 2024Task Singular Vectors: Reducing Task Interference in Model Mergingcited by 143
  9. 2022FacT: Factor-Tuning for Lightweight Adaptation on Vision Transformercited by 133
  10. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  11. 2025No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspacescited by 90
  12. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 83
38 more
  1. 2021Vision Transformer Pruningcited by 77
  2. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 70
  3. 2022SGD with Large Step Sizes Learns Sparse Featurescited by 66
  4. 2022Compressing Volumetric Radiance Fields to 1 MBcited by 66
  5. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 64
  6. 2021Towards Mixed-Precision Quantization of Neural Networks via Constrained Optimizationcited by 57
  7. 2022Implicit Regularization Towards Rank Minimization in ReLU Networkscited by 53
  8. 2021A Survey on Green Deep Learningcited by 52
  9. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  10. 2022Singular Value Fine-tuning: Few-shot Segmentation requires Few-parameters Fine-tuningcited by 50
  11. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 42
  12. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  13. 2021Revisiting Dynamic Convolution via Matrix Decompositioncited by 35
  14. 2022Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharingcited by 35
  15. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  16. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  17. 2021DECORE: Deep Compression with Reinforcement Learningcited by 29
  18. 2023Tensor Networks Meet Neural Networks: A Survey and Future Perspectivescited by 29
  19. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  20. 2021Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Schemecited by 23
  21. 2022Data-Efficient Structured Pruning via Submodular Optimizationcited by 22
  22. 2023BiBench: Benchmarking and Analyzing Network Binarizationcited by 22
  23. 2021ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximationscited by 21
  24. 2021Distilling a Powerful Student Model via Online Knowledge Distillationcited by 21
  25. 2021L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimizationcited by 21
  26. 2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
  27. 2023HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networkscited by 20
  28. 2024Faster and Lighter LLMs: A Survey on Current Challenges and Way Forwardcited by 20
  29. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  30. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 19
  31. 2023A Rainbow in Deep Network Black Boxescited by 18
  32. 2022SGD and Weight Decay Secretly Minimize the Rank of Your Neural Networkcited by 17
  33. 2023Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AIcited by 17
  34. 2021Network Pruning using Adaptive Exemplar Filterscited by 16
  35. 2022Compression-aware Training of Neural Networks using Frank-Wolfecited by 16
  36. 2022Exploring Extreme Parameter Compression for Pre-trained Language Modelscited by 15
  37. 2021Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsitycited by 14
  38. 2021Convolutional Neural Network Compression through Generalized Kronecker Product Decompositioncited by 14

50 most cited of 304 All 304 (JSON)