Data to 14 Oct 2026

Dynamic Network Surgery for Efficient DNNs

arXiv 1608.04493 ↗First posted 16 Aug 2016cs.NE cs.CV cs.LGcited by 233

Not on the map. Why?

Abstract (arXiv)

Deep learning has become a ubiquitous technology to improve machine intelligence. However, most of the existing deep models are structurally very complex, making them difficult to be deployed on the mobile platforms with limited computational power. In this paper, we propose a novel network compression method called dynamic network surgery, which can remarkably reduce the network complexity by making on-the-fly connection pruning. Unlike the previous methods which accomplish this task in a greedy way, we properly incorporate connection splicing into the whole process to avoid incorrect pruning and make it as a continual network maintenance. The effectiveness of our method is proved with experiments. Without any accuracy loss, our method can efficiently compress the number of parameters in LeNet-5 and AlexNet by a factor of bm108× and bm17.7× respectively, proving that it outperforms the recent pruning method by considerable margins. Code and some models are available at https://github.com/yiwenguo/Dynamic-Network-Surgery.

Key facts. Cited by 233 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 66; 2022: 51; 2023: 45; 2024: 30; 2025: 31; 2026: 10.

233 papers.

Sort:

By area:

2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
2023DepGraph: Towards Any Structural Pruningcited by 203
2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
2023Continual Pre-training of Language Modelscited by 198
2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
2021Group Fisher Pruning for Practical Network Compressioncited by 131
2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 110
2021Distilling Object Detectors via Decoupled Featurescited by 106
2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 79
2021CHIP: CHannel Independence-based Pruning for Compact Neural Networkscited by 78

38 more

2024MagMax: Leveraging Model Merging for Seamless Continual Learningcited by 67
2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 53
2021Dynamic Sparse Training for Deep Reinforcement Learningcited by 49
2024ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Cachingcited by 48
2021Differentiable Subset Pruning of Transformer Headscited by 47
2021The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translationcited by 47
2021Effective Sparsification of Neural Networks with Global Sparsity Constraintcited by 45
2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 43
2021Recent Advances on Neural Network Pruning at Initializationcited by 41
2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
2021Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Searchcited by 34
2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
2021Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learningcited by 30
2021Efficient Neural Network Training via Forward and Backward Propagation Sparsificationcited by 30
2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
2023NTK-SAP: Improving neural network pruning by aligning training dynamicscited by 30
2023AutoReP: Automatic ReLU Replacement for Fast Private Network Inferencecited by 29
2021Effective Model Sparsification by Scheduled Grow-and-Prune Methodscited by 26
2021An Information-Theoretic Justification for Model Pruningcited by 25
2022Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraintscited by 25
2024SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Modelscited by 25
2023Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reductioncited by 24
2023Adapting a Language Model While Preserving its General Knowledgecited by 23
2022Fairify: Fairness Verification of Neural Networkscited by 22
2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 22
2022Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptionscited by 20
2021Layer-wise Model Pruning based on Mutual Informationcited by 19
2022OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantizationcited by 19
2024Effective Interplay between Sparsity and Quantization: From Theory to Practicecited by 19
2024ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Modelscited by 19
2021Stealthy Backdoors as Compression Artifactscited by 18
2024Progressive Gradient Flow for Robust N:M Sparsity Training in Transformerscited by 18
2024OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimizationcited by 18
2021Network Pruning using Adaptive Exemplar Filterscited by 16

50 most cited of 233 All 233 (JSON)