mlsanityData to 7 Oct 2026

Dynamic Model Pruning with Feedback

arXiv 2006.07253 ↗First posted 12 Jun 2020cs.LG stat.MLcited by 125

Not on the map. Why?

Abstract (arXiv)

Deep neural networks often have millions of parameters. This can hinder their deployment to low-end devices, not only due to high memory requirements but also because of increased latency at inference. We propose a novel model compression method that generates a sparse trained model without additional overhead: by allowing (i) dynamic allocation of the sparsity pattern and (ii) incorporating feedback signal to reactivate prematurely pruned weights we obtain a performant sparse model in one single training pass (retraining is not needed, but can further improve the performance). We evaluate our method on CIFAR-10 and ImageNet, and show that the obtained sparse models can reach the state-of-the-art performance of dense models. Moreover, their performance surpasses that of models generated by all previously proposed pruning schemes.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 31; 2022: 21; 2023: 32; 2024: 21; 2025: 17; 2026: 3.

125 papers.

Sort:

By area:

  1. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  2. 2021Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuningcited by 145
  3. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  4. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 83
  5. 2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 76
  6. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simplecited by 59
  7. 2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 58
  8. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  9. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  10. 2022Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Modelscited by 42
  11. 2023The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Mattercited by 42
  12. 2023Scaling Laws for Sparsely-Connected Foundation Modelscited by 41
38 more
  1. 2021Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decompositioncited by 33
  2. 2021On the Existence of Universal Lottery Ticketscited by 30
  3. 2021GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarizationcited by 29
  4. 2022An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformerscited by 28
  5. 2023NOLA: Compressing LoRA using Linear Combination of Random Basiscited by 28
  6. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  7. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  8. 2023Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!cited by 27
  9. 2021Understanding the Dynamics of DNNs Using Graph Modularitycited by 26
  10. 2021Time-Correlated Sparsification for Communication-Efficient Federated Learningcited by 25
  11. 2021Network Pruning That Matters: A Case Study on Retraining Variantscited by 25
  12. 2021Effective Model Sparsification by Scheduled Grow-and-Prune Methodscited by 25
  13. 2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
  14. 2021The Elastic Lottery Ticket Hypothesiscited by 23
  15. 2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
  16. 2023Sparse Model Soups: A Recipe for Improved Pruning via Model Averagingcited by 20
  17. 2021RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networkscited by 19
  18. 2021PQK: Model Compression via Pruning, Quantization, and Knowledge Distillationcited by 19
  19. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  20. 20211xN Pattern for Pruning Convolutional Neural Networkscited by 16
  21. 2022Pruning Networks with Cross-Layer Ranking & k-Reciprocal Nearest Filterscited by 16
  22. 2022Compression-aware Training of Neural Networks using Frank-Wolfecited by 16
  23. 2023ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasetscited by 16
  24. 2021Masked Training of Neural Networks with Partial Gradientscited by 15
  25. 2021Differentiable Network Pruning for Microcontrollerscited by 15
  26. 2023Efficient Joint Optimization of Layer-Adaptive Weight Pruning in Deep Neural Networkscited by 15
  27. 2022Spatial Mixture-of-Expertscited by 14
  28. 2023PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMscited by 14
  29. 2022Is Complexity Required for Neural Network Pruning? A Case Study on Global Magnitude Pruningcited by 13
  30. 2023Quantized Distributed Training of Large Models with Convergence Guaranteescited by 13
  31. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  32. 2021How I Learned to Stop Worrying and Love Retrainingcited by 12
  33. 2022Soft Masking for Cost-Constrained Channel Pruningcited by 12
  34. 2024ELRT: Efficient Low-Rank Training for Compact Convolutional Neural Networkscited by 11
  35. 2024SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Modelscited by 11
  36. 2022UDC: Unified DNAS for Compressible TinyML Modelscited by 10
  37. 2023Adaptive Federated Pruning in Hierarchical Wireless Networkscited by 10
  38. 2023Differentiable Transportation Pruningcited by 10

50 most cited of 125 All 125 (JSON)