mlsanityData to 7 Oct 2026

Convolutional Neural Network Pruning with Structural Redundancy Reduction

arXiv 2104.03438 ↗First posted 8 Apr 2021cs.CV cs.LGcited by 73

Not on the map. Why?

Abstract (arXiv)

Convolutional neural network (CNN) pruning has become one of the most successful network compression approaches in recent years. Existing works on network pruning usually focus on removing the least important filters in the network to achieve compact architectures. In this study, we claim that identifying structural redundancy plays a more essential role than finding unimportant filters, theoretically and empirically. We first statistically model the network pruning problem in a redundancy reduction perspective and find that pruning in the layer(s) with the most structural redundancy outperforms pruning the least important filters across all layers. Based on this finding, we then propose a network pruning approach that identifies structural redundancy of a CNN and prunes filters in the selected layer(s) with the most redundancy. Experiments on various benchmark network architectures and datasets show that our proposed approach significantly outperforms the previous state-of-the-art.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). Builds on 36 of its 53 references that are in the map. First posted on arXiv 8 Apr 2021.

Builds on

36 of 53 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,218
  10. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  11. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  12. 2018Rethinking the Value of Network Pruningcited by 595
24 more
  1. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  2. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  3. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  4. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  5. 2018Slimmable Neural Networkscited by 301
  6. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  7. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  8. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  9. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  10. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  11. 2015Compressing Neural Networks with the Hashing Trickcited by 150
  12. 2019Towards Optimal Structured CNN Pruning via Generative Adversarial Learningcited by 145
  13. 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 118
  14. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  15. 2020Channel Pruning via Automatic Structure Searchcited by 84
  16. 2019Network Pruning via Transformable Architecture Searchcited by 80
  17. 2018AutoPruner: An End-to-End Trainable Filter Pruning Method for Efficient Deep Model Inferencecited by 77
  18. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  19. 2019Approximated Oracle Filter Pruning for Destructive CNN Width Optimizationcited by 35
  20. 2020TRP: Trained Rank Pruning for Efficient Deep Neural Networkscited by 32
  21. 2018Efficient Sparse-Winograd Convolutional Neural Networkscited by 20
  22. 2019CUP: Cluster Pruning for Compressing Deep Neural Networkscited by 7
  23. 2018Recovering from Random Pruning: On the Plasticity of Deep Convolutional Neural Networkscited by 6
  24. 2019Online Filter Clustering and Pruning for Efficient Convnetscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 12; 2023: 15; 2024: 16; 2025: 11; 2026: 4.

73 papers.

Sort:

By area:

  1. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  2. 2021Zero-Shot Knowledge Distillation from a Decision-Based Black-Box Modelcited by 37
  3. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  4. 2023Exploring Incompatible Knowledge Transfer in Few-shot Image Generationcited by 19
  5. 2022Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNscited by 16
  6. 2022CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Spacecited by 13
  7. 2023Geometry-aware training of factorized layers in tensor Tucker formatcited by 12
  8. 2021SPATL: Salient Parameter Aggregation and Transfer Learning for Heterogeneous Clients in Federated Learningcited by 11
  9. 2023Differentiable Transportation Pruningcited by 10
  10. 2022SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevancecited by 9
  11. 2024Direct Distillation between Different Domainscited by 7
  12. 2025"X of Information'' Continuum: A Survey on AI-Driven Multi-dimensional Metrics for Next-Generation Networked Systemscited by 7
38 more
  1. 2022Ensemble Knowledge Guided Sub-network Search and Fine-tuning for Filter Pruningcited by 5
  2. 2025From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Modelscited by 5
  3. 2021Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximationcited by 4
  4. 2022Network Pruning via Feature Shift Minimizationcited by 4
  5. 2024RepNeXt: A Fast Multi-Scale CNN using Structural Reparameterizationcited by 4
  6. 2024GSSF: Generalized Structural Sparse Function for Deep Cross-modal Metric Learningcited by 4
  7. 2025Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Modelscited by 4
  8. 2021Channel redundancy and overlap in convolutional neural networks with channel-wise NNK graphscited by 3
  9. 2021Stacked BNAS: Rethinking Broad Convolutional Neural Network for Neural Architecture Searchcited by 3
  10. 2022LOFT: Finding Lottery Tickets through Filter-wise Trainingcited by 3
  11. 2024Learning Minimal Neural Specificationscited by 3
  12. 2024ONNXPruner: ONNX-Based General Model Pruning Adaptercited by 3
  13. 2024TropNNC: Structured Neural Network Compression Using Tropical Geometrycited by 3
  14. 2025One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compressioncited by 3
  15. 2021Bayesian Optimization with Clustering and Rollback for CNN Auto Pruningcited by 2
  16. 2021Batch Normalization Tells You Which Filter is Importantcited by 2
  17. 2022InDistill: Information flow-preserving knowledge distillation for model compressioncited by 2
  18. 2022CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Executioncited by 2
  19. 2022Exploring Generalizable Distillation for Efficient Medical Image Segmentationcited by 2
  20. 2023Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Samplercited by 2
  21. 2023Learning Kernel-Modulated Neural Representation for Efficient Light Field Compressioncited by 2
  22. 2024Efficient and Flexible Method for Reducing Moderate-size Deep Neural Networks with Condensationcited by 2
  23. 2024Enhancing Accuracy and Parameter-Efficiency of Neural Representations for Network Parameterizationcited by 2
  24. 2026Smart Scissor: Coupling Spatial Redundancy Reduction and CNN Compression for Embedded Hardwarecited by 2
  25. 2021Pruning with Compensation: Efficient Channel Pruning for Deep Convolutional Neural Networkscited by 1
  26. 2021Learning Pruned Structure and Weights Simultaneously from Scratch: an Attention based Approachcited by 1
  27. 2021SNF: Filter Pruning via Searching the Proper Number of Filterscited by 1
  28. 2023Training Strategies for Vision Transformers for Object Detectioncited by 1
  29. 2023A Differentiable Framework for End-to-End Learning of Hybrid Structured Compressioncited by 1
  30. 2023Find the Lady: Permutation and Re-Synchronization of Deep Neural Networkscited by 1
  31. 2024Computer Vision Model Compression Techniques for Embedded Systems: A Surveycited by 1
  32. 2025Enhanced Structured Lasso Pruning with Class-wise Informationcited by 1
  33. 2025Structure-Aware Automatic Channel Pruning by Searching with Graph Embeddingcited by 1
  34. 2025Hardware-Aware YOLO Compression for Low-Power Edge AI on STM32U5 for Weeds Detection in Digital Agriculturecited by 1
  35. 2021Basis Scaling and Double Pruning for Efficient Inference in Network-Based Transfer Learningcited by 0
  36. 2021AIP: Adversarial Iterative Pruning Based on Knowledge Transfer for Convolutional Neural Networkscited by 0
  37. 2021Joint Channel and Weight Pruning for Model Acceleration on Moblie Devicescited by 0
  38. 2021Class-Discriminative CNN Compressioncited by 0

50 most cited of 73 All 73 (JSON)