mlsanityData to 7 Oct 2026

EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis

arXiv 1905.05934 ↗First posted 15 May 2019cs.LG stat.MLcited by 98

Not on the map. Why?

Abstract (arXiv)

Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices. To achieve this goal, we introduce a novel network reparameterization based on the Kronecker-factored eigenbasis (KFE), and then apply Hessian-based structured pruning methods in this basis. As opposed to existing Hessian-based pruning algorithms which do pruning in parameter coordinates, our method works in the KFE where different weights are approximately independent, enabling accurate pruning and fast computation. We demonstrate empirically the effectiveness of the proposed method through extensive experiments. In particular, we highlight that the improvements are especially significant for more challenging datasets and networks. With negligible loss of accuracy, an iterative-pruning version gives a 10× reduction in model size and a 8× reduction in FLOPs on wide ResNet32.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 15; 2023: 16; 2024: 21; 2025: 24; 2026: 8.

98 papers.

Sort:

By area:

  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  2. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  3. 2021Knowledge distillation: A good teacher is patient and consistentcited by 255
  4. 2023DepGraph: Towards Any Structural Pruningcited by 202
  5. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  6. 2023Structured Pruning for Deep Convolutional Neural Networks: A surveycited by 122
  7. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  8. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  9. 2021M-FAC: Efficient Matrix-Free Approximations of Second-Order Informationcited by 54
  10. 2024Efficient Expert Pruning for Sparse Mixture-of-Experts Language Models: Enhancing Performance and Reducing Inference Costscited by 51
  11. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  12. 2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 45
38 more
  1. 2021Hessian-Aware Pruning and Optimal Neural Implantcited by 43
  2. 2021Recent Advances on Neural Network Pruning at Initializationcited by 41
  3. 2022The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networkscited by 41
  4. 2023The LLM Surgeoncited by 36
  5. 2022Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradientscited by 34
  6. 2021Applications and Techniques for Fast Machine Learning in Sciencecited by 32
  7. 2022Pruning's Effect on Generalization Through the Lens of Training and Regularizationcited by 31
  8. 2021SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruningcited by 30
  9. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  10. 2022Dual Lottery Ticket Hypothesiscited by 27
  11. 2023Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!cited by 27
  12. 2023Federated Domain Generalization: A Surveycited by 25
  13. 2022Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Networkcited by 21
  14. 2022Winning the Lottery Ahead of Time: Efficient Early Network Pruningcited by 21
  15. 2022CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Modelscited by 20
  16. 2021Scaling Up Exact Neural Network Compression by ReLU Stabilitycited by 19
  17. 2023Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruningcited by 16
  18. 2024Using Degeneracy in the Loss Landscape for Mechanistic Interpretabilitycited by 12
  19. 2023Automatic Attention Pruning: Improving and Automating Model Pruning using Attentionscited by 11
  20. 2024Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruningcited by 10
  21. 2024Preserving Deep Representations In One-Shot Pruning: A Hessian-Free Second-Order Optimization Frameworkcited by 10
  22. 2021Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNNcited by 9
  23. 2022Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithmcited by 9
  24. 2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
  25. 2023Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFACcited by 9
  26. 2025TeleSparse: Practical Privacy-Preserving Verification of Deep Neural Networkscited by 9
  27. 2023Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regionscited by 8
  28. 2023Large Multimodal Model Compression via Efficient Pruning and Distillation at AntGroupcited by 8
  29. 2024Towards Meta-Pruning via Optimal Transportcited by 8
  30. 2025MergeMoE: Efficient Compression of MoE Models via Expert Output Mergingcited by 8
  31. 2024Dynamic Sparse Learning: A Novel Paradigm for Efficient Recommendationcited by 7
  32. 2024Is Oracle Pruning the True Oracle?cited by 7
  33. 2025QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edgecited by 7
  34. 2025GradMetaNet: An Equivariant Architecture for Learning on Gradientscited by 7
  35. 2023Error Feedback Can Accurately Compress Preconditionerscited by 6
  36. 2025Masking Teacher and Reinforcing Student for Distilling Vision-Language Modelscited by 6
  37. 2026On State Reduction in Linear Attentioncited by 6
  38. 2024Mutual Information Preserving Neural Network Pruningcited by 5

50 most cited of 98 All 98 (JSON)