mlsanityData to 7 Oct 2026

RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Merging

arXiv 2110.01397 ↗First posted 30 Sep 2021cs.LG cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Pruning Deep Neural Networks (DNNs) is a prominent field of study in the goal of inference runtime acceleration. In this paper, we introduce a novel data-free pruning protocol RED++. Only requiring a trained neural network, and not specific to DNN architecture, we exploit an adaptive data-free scalar hashing which exhibits redundancies among neuron weight values. We study the theoretical and empirical guarantees on the preservation of the accuracy from the hashing as well as the expected pruning ratio resulting from the exploitation of said redundancies. We propose a novel data-free pruning technique of DNN layers which removes the input-wise redundant operations. This algorithm is straightforward, parallelizable and offers novel perspective on DNN pruning by shifting the burden of large computation to efficient memory access and allocation. We provide theoretical guarantees on RED++ performance and empirically demonstrate its superiority over other data-free pruning methods and its competitiveness with data-driven ones on ResNets, MobileNets and EfficientNets.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 52 of its 71 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

52 of 71 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2017Mask R-CNNcited by 7,259
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  6. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  8. 2016Wide Residual Networkscited by 3,057
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  11. 2021Going deeper with Image Transformerscited by 759
  12. 2018Rethinking the Value of Network Pruningcited by 595
38 more
  1. 2020What is the State of Neural Network Pruning?cited by 578
  2. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  3. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  4. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  5. 2019The State of Sparsity in Deep Neural Networkscited by 471
  6. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  7. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  8. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  9. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  10. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  11. 2017A Survey of Model Compression and Acceleration for Deep Neural Networkscited by 298
  12. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  13. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  14. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  15. 2021Accelerating Sparse Deep Neural Networkscited by 213
  16. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  17. 2017Deep Rewiring: Training very sparse deep networkscited by 136
  18. 2020Dynamic Model Pruning with Feedbackcited by 125
  19. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  20. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  21. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  22. 2020SCOP: Scientific Control for Reliable Neural Network Pruningcited by 92
  23. 2019And the Bit Goes Down: Revisiting the Quantization of Neural Networkscited by 78
  24. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  25. 2020Good Subnetworks Provably Exist: Pruning via Greedy Forward Selectioncited by 70
  26. 2020Lookahead: A Far-Sighted Alternative of Magnitude-based Pruningcited by 61
  27. 2020Discrimination-aware Network Pruning for Deep Model Compressioncited by 50
  28. 2019PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devicescited by 48
  29. 2020Pruning Filter in Filtercited by 46
  30. 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45
  31. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  32. 2020Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsitycited by 30
  33. 2020Neuron Merging: Compensating for Pruned Neuronscited by 24
  34. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  35. 2020Layer-Wise Data-Free CNN Compressioncited by 13
  36. 2018Escoin: Efficient Sparse Convolutional Neural Network Inference on GPUscited by 11
  37. 2020Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)cited by 6
  38. 2021Network Pruning via Resource Reallocationcited by 6

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 8; 2024: 4; 2025: 3; 2026: 3.

19 papers.

Sort:

By area:

  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  2. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  3. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  4. 2024Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envisioncited by 5
  5. 2026AIMER: Calibration-Free Task-Agnostic MoE Expert Pruningcited by 5
  6. 2022Matching DNN Compression and Cooperative Training with Resources and Data Availabilitycited by 4
  7. 2023Data-Free Dynamic Compression of CNNs for Tractable Efficiencycited by 4
  8. 2023Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applicationscited by 3
  9. 2025Collaborative Compression for Large-Scale MoE Deployment on Edgecited by 3
  10. 2024STAT: Shrinking Transformers After Trainingcited by 2
  11. 2025Training-Free Restoration of Pruned Neural Networkscited by 2
  12. 2023Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappingscited by 1
7 more
  1. 2023Designing strong baselines for ternary neural network quantization through support and mass equalizationcited by 0
  2. 2023Archtree: on-the-fly tree-structured exploration for latency-aware pruning of deep neural networkscited by 0
  3. 2023Optimizing Convolutional Neural Network Architecturecited by 0
  4. 2024A Comprehensive Study of Structural Pruning for Vision Modelscited by 0
  5. 2025IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruningcited by 0
  6. 2026Privacy-Preserving Model Transcription with Differentially Private Synthetic Distillationcited by 0
  7. 2026HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networkscited by 0