mlsanityData to 7 Oct 2026

Data-Free Dynamic Compression of CNNs for Tractable Efficiency

arXiv 2309.17211 ↗First posted 29 Sep 2023cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

To reduce the computational cost of convolutional neural networks (CNNs) on resource-constrained devices, structured pruning approaches have shown promise in lowering floating-point operations (FLOPs) without substantial drops in accuracy. However, most methods require fine-tuning or specific training procedures to achieve a reasonable trade-off between retained accuracy and reduction in FLOPs, adding computational overhead and requiring training data to be available. To this end, we propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning. Our approach utilizes locality-sensitive hashing (LSH) to detect redundancies in the channel dimension of latent feature maps, compressing similar channels to reduce input and filter depth simultaneously, resulting in cheaper convolutions. We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy by swapping the convolution modules in a ResNet34 on CIFAR-10 for our HASTE module.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

41 of 58 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2022A ConvNet for the 2020scited by 3,545
  12. 2020Reformer: The Efficient Transformercited by 1,658
29 more
  1. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  2. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  4. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  5. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  6. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  7. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  8. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  10. 2019GhostNet: More Features from Cheap Operationscited by 410
  11. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  12. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  13. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  14. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 148
  15. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  16. 2018Dynamic Channel Pruning: Feature Boosting and Suppressioncited by 129
  17. 2015Accelerating Very Deep Convolutional Networks for Classification and Detectioncited by 109
  18. 2019Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inferencecited by 91
  19. 2018Channel Gating Neural Networkscited by 80
  20. 2017More is Less: A More Complicated Network with Less Inference Complexitycited by 66
  21. 2019SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systemscited by 46
  22. 2019Batch-Shaping for Learning Conditional Channel Gated Networkscited by 42
  23. 2021Fire Together Wire Together: A Dynamic Pruning Approach with Self-Supervised Mask Predictioncited by 24
  24. 2023Unified Data-Free Compression: Pruning and Quantization without Fine-Tuningcited by 22
  25. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  26. 2018Dynamic Sparse Graph for Efficient Deep Learningcited by 18
  27. 2017Memory-Efficient Implementation of DenseNetscited by 17
  28. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  29. 2021Efficient Inference via Universal LSH Kernelcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Efficient Data Driven Mixture-of-Expert Extraction from Trained Networkscited by 3
  2. 2025Variance-Based Pruning for Accelerating and Compressing Trained Networkscited by 3
  3. 2025PROM: Prioritize Reduction of Multiplications Over Lower Bit-Widths for Efficient CNNscited by 1
  4. 2026HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networkscited by 0