mlsanityData to 7 Oct 2026

DepGraph: Towards Any Structural Pruning

arXiv 2301.12900 ↗First posted 30 Jan 2023cs.AI cs.CVcited by 202

Not on the map. Why?

Abstract (arXiv)

Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks. However, the parameter-grouping patterns vary widely across different models, making architecture-specific pruners, which rely on manually-designed grouping schemes, non-generalizable to new architectures. In this work, we study a highly-challenging yet barely-explored task, any structural pruning, to tackle general structural pruning of arbitrary architecture like CNNs, RNNs, GNNs and Transformers. The most prominent obstacle towards this goal lies in the structural coupling, which not only forces different layers to be pruned simultaneously, but also expects all removed parameters to be consistently unimportant, thereby avoiding structural issues and significant performance degradation after pruning. To address this problem, we propose a general and fully automatic method, Dependency Graph (DepGraph), to explicitly model the dependency between layers and comprehensively group coupled parameters for pruning. In this work, we extensively evaluate our method on several architectures and tasks, including ResNe(X)t, DenseNet, MobileNet and Vision transformer for images, GAT for graph, DGCNN for 3D point cloud, alongside LSTM for language, and demonstrate that, even with a simple norm-based criterion, the proposed method consistently yields gratifying performances.

Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). Builds on 63 of its 79 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

63 of 79 references.

References · 63

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2017Graph Attention Networkscited by 8,854
  6. 2016Densely Connected Convolutional Networkscited by 6,691
  7. 2017Inductive Representation Learning on Large Graphscited by 6,284
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2014Going Deeper with Convolutionscited by 4,231
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  11. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  12. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
38 more
  1. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  4. 2020Designing Network Design Spacescited by 954
  5. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  6. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  7. 2019Importance Estimation for Neural Network Pruningcited by 531
  8. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  9. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 398
  10. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  11. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 365
  12. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 313
  13. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 291
  14. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  15. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  16. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  17. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  18. 2019MetaPruning: Meta Learning for Automatic Neural Network Channel Pruningcited by 226
  19. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  20. 2015Quantized Convolutional Neural Networks for Mobile Devicescited by 212
  21. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  22. 2018NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applicationscited by 146
  23. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  24. 2020Distilling Knowledge from Graph Convolutional Networkscited by 128
  25. 2023Dataset Distillation: A Comprehensive Reviewcited by 126
  26. 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 122
  27. 2019Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networkscited by 118
  28. 2022Dataset Distillation via Factorizationcited by 115
  29. 2018Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layerscited by 112
  30. 2019Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networkscited by 103
  31. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  32. 2020Neural Pruning via Growing Regularizationcited by 97
  33. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  34. 2020ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgettingcited by 85
  35. 2019Network Pruning via Transformable Architecture Searchcited by 80
  36. 2022Deep Model Reassemblycited by 78
  37. 2019Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structurecited by 76
  38. 2019Towards Efficient Model Compression via Learned Global Rankingcited by 75

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 37; 2024: 65; 2025: 63; 2026: 37.

202 papers.

Sort:

By area:

  1. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  2. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  3. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  4. 2023A Survey on Model Compression for Large Language Modelscited by 263
  5. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  6. 2023Structural Pruning for Diffusion Modelscited by 193
  7. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  8. 2023Diffusion Model as Representation Learnercited by 95
  9. 2025Efficient Reasoning Models: A Surveycited by 93
  10. 2024Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methodscited by 80
  11. 2023GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graphcited by 69
  12. 2023Priority-Centric Human Motion Generation in Discrete Latent Spacecited by 66
38 more
  1. 2024LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insightscited by 65
  2. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  3. 2023SG-Former: Self-guided Transformer with Evolving Token Reallocationcited by 32
  4. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 32
  5. 2023Denoising Diffusion Step-aware Modelscited by 24
  6. 2023Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspectivecited by 23
  7. 2025Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingcited by 23
  8. 2023Deep Graph Reprogrammingcited by 22
  9. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  10. 2024Isomorphic Pruning for Vision Modelscited by 21
  11. 2023Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Modelscited by 20
  12. 2023SlimSAM: 0.1% Data Makes Segment Anything Slimcited by 20
  13. 2024Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratchcited by 20
  14. 2023ECoFLaP: Efficient Coarse-to-Fine Layer-Wise Pruning for Vision-Language Modelscited by 19
  15. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  16. 2024SparseDM: Toward Sparse Efficient Diffusion Modelscited by 18
  17. 2023Segment Anything in Non-Euclidean Domains: Challenges and Opportunitiescited by 16
  18. 2024Superficial Safety Alignment Hypothesiscited by 14
  19. 2025Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probingcited by 14
  20. 2025Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compressioncited by 14
  21. 2023BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusioncited by 13
  22. 2024Teddy: Efficient Large-Scale Dataset Distillation via Taylor-Approximated Matchingcited by 13
  23. 2025On-Device Diffusion Transformer Policy for Efficient Robot Manipulationcited by 13
  24. 2024AMD: Automatic Multi-step Distillation of Large-scale Vision Modelscited by 12
  25. 2023Accelerating Deep Neural Networks via Semi-Structured Activation Sparsitycited by 10
  26. 2024S2FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsitycited by 10
  27. 2025ToMoE: Converting Dense Large Language Models to Mixture-of-Experts through Dynamic Structural Pruningcited by 10
  28. 2023Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfercited by 9
  29. 2023RefConv: Re-parameterized Refocusing Convolution for Powerful ConvNetscited by 9
  30. 2023Class-Aware Pruning for Efficient Neural Networkscited by 9
  31. 2024Neural Lineagecited by 9
  32. 2024Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compressioncited by 8
  33. 2024Structurally Prune Anything: Any Architecture, Any Framework, Any Timecited by 8
  34. 2024Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retrainingcited by 8
  35. 2024Individual Content and Motion Dynamics Preserved Pruning for Video Diffusion Modelscited by 8
  36. 2025MergeMoE: Efficient Compression of MoE Models via Expert Output Mergingcited by 8
  37. 2024Shaving Weights with Occam's Razor: Bayesian Sparsification for Neural Networks Using the Marginal Likelihoodcited by 7
  38. 2024Is Oracle Pruning the True Oracle?cited by 7

50 most cited of 202 All 202 (JSON)