mlsanityData to 7 Oct 2026

Neural Network Compression via Effective Filter Analysis and Hierarchical Pruning

arXiv 2206.03596 ↗First posted 7 Jun 2022cs.LG cs.CV eess.IVcited by 0

Not on the map. Why?

Abstract (arXiv)

Network compression is crucial to making the deep networks to be more efficient, faster, and generalizable to low-end hardware. Current network compression methods have two open problems: first, there lacks a theoretical framework to estimate the maximum compression rate; second, some layers may get over-prunned, resulting in significant network performance drop. To solve these two problems, this study propose a gradient-matrix singularity analysis-based method to estimate the maximum network redundancy. Guided by that maximum rate, a novel and efficient hierarchical network pruning algorithm is developed to maximally condense the neuronal network structure without sacrificing network performance. Substantial experiments are performed to demonstrate the efficacy of the new method for pruning several advanced convolutional neural network (CNN) architectures. Compared to existing pruning methods, the proposed pruning algorithm achieved state-of-the-art performance. At the same or similar compression ratio, the new method provided the highest network prediction accuracy as compared to other methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

28 of 40 references.

References · 28

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2014Going Deeper with Convolutionscited by 4,231
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  11. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  12. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
16 more
  1. 2019Importance Estimation for Neural Network Pruningcited by 531
  2. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  3. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  4. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  5. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  6. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  7. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  8. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  9. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  10. 2019Towards Optimal Structured CNN Pruning via Generative Adversarial Learningcited by 145
  11. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  12. 2019Global Sparse Momentum SGD for Pruning Very Deep Neural Networkscited by 70
  13. 2017An Entropy-based Pruning Method for CNN Compressioncited by 37
  14. 2014Efficient and Accurate Approximations of Nonlinear Convolutional Networkscited by 30
  15. 2018Filter Distillation for Network Compressioncited by 17
  16. 2018Exploring Linear Relationship in Feature Map Subspace for ConvNets Compressioncited by 11

Built on by

Not cited yet.