mlsanityData to 14 Oct 2026

Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining

arXiv 2407.19126 ↗First posted 26 Jul 2024cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

To remove redundant components of large language models (LLMs) without incurring significant computational costs, this work focuses on single-shot pruning without a retraining phase. We simplify the pruning process for Transformer-based LLMs by identifying a depth-2 pruning structure that functions independently. Additionally, we propose two inference-aware pruning criteria derived from the optimization perspective of output approximation, which outperforms traditional training-aware metrics such as gradient and Hessian. We also introduce a two-step reconstruction technique to mitigate pruning errors without model retraining. Experimental results demonstrate that our approach significantly reduces computational costs and hardware requirements while maintaining superior performance across various datasets and models.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 46 of its 61 references that are in the map. First posted on arXiv 26 Jul 2024.

Builds on

46 of 61 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  11. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
34 more
  1. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  4. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  5. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  6. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  7. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  8. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  9. 2019Importance Estimation for Neural Network Pruningcited by 541
  10. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 527
  11. 2019The State of Sparsity in Deep Neural Networkscited by 476
  12. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 442
  13. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  14. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  15. 2018Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Accelerationcited by 367
  16. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  17. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  18. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 221
  19. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  20. 2019Structured Pruning of Large Language Modelscited by 203
  21. 2023DepGraph: Towards Any Structural Pruningcited by 203
  22. 2020Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learningcited by 185
  23. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  24. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  25. 2023Fluctuation-based Adaptive Structured Pruning for Large Language Modelscited by 111
  26. 2024Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methodscited by 82
  27. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 49
  28. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  29. 2018Filter Distillation for Network Compressioncited by 17
  30. 2024The Need for Speed: Pruning Transformers with One Recipecited by 16
  31. 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
  32. 2018Full deep neural network training on a pruned weight budgetcited by 11
  33. 2023Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Modelscited by 7
  34. 2023Breaking through Deterministic Barriers: Randomized Pruning Mask Generation and Selectioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 2.

8 papers.

  1. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 67
  2. 2024Superficial Safety Alignment Hypothesiscited by 14
  3. 2026Purifying Generative LLMs from Backdoors without Prior Knowledge or Clean Referencecited by 7
  4. 2025Discovering Transformer Circuits via a Hybrid Attribution and Pruning Frameworkcited by 4
  5. 2025WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inferencecited by 3
  6. 2025SEAP: Training-free Sparse Expert Activation Pruning Unlock the Brainpower of Large Language Modelscited by 2
  7. 2025SimLens for Early Exit in Large Language Models: Eliciting Accurate Latent Predictions with One More Tokencited by 0
  8. 2026Position: Retire the "Positive Backdoor" Label -- Secret Alignment Requires Strict and Systematic Evaluationcited by 0