mlsanityData to 14 Oct 2026

One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models

arXiv 2310.09499 ↗First posted 14 Oct 2023cs.CL cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

Various Large Language Models~(LLMs) from the Generative Pretrained Transformer(GPT) family have achieved outstanding performances in a wide range of text generation tasks. However, the enormous model sizes have hindered their practical use in real-world applications due to high inference latency. Therefore, improving the efficiencies of LLMs through quantization, pruning, and other means has been a key issue in LLM studies. In this work, we propose a method based on Hessian sensitivity-aware mixed sparsity pruning to prune LLMs to at least 50% sparsity without the need of any retraining. It allocates sparsity adaptively based on sensitivity, allowing us to reduce pruning-induced error while maintaining the overall sparsity level. The advantages of the proposed method exhibit even more when the sparsity is extremely high. Furthermore, our method is compatible with quantization, enabling further compression of LLMs. We have released the available code.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 29 of its 37 references that are in the map. First posted on arXiv 14 Oct 2023.

Builds on

29 of 37 references.

References · 29

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  11. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  12. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
17 more
  1. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  2. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 508
  3. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  4. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  5. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  6. 2021Post-Training Quantization for Vision Transformercited by 278
  7. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
  8. 2019Structured Pruning of Large Language Modelscited by 203
  9. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 178
  10. 2021Block Pruning For Faster Transformerscited by 177
  11. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  12. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  13. 2020Degree-Quant: Quantization-Aware Training for Graph Neural Networkscited by 96
  14. 2021Prune Once for All: Sparse Pre-Trained Language Modelscited by 66
  15. 2020Contrastive Distillation on Intermediate Representations for Language Model Compressioncited by 57
  16. 2018A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Datasetcited by 55
  17. 2020Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domainscited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 6; 2026: 1.

16 papers.

Sort:

By area:

  1. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  2. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  3. 2024Faster and Lighter LLMs: A Survey on Current Challenges and Way Forwardcited by 21
  4. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 16
  5. 2024Slice-Level Scheduling for High Throughput and Load Balanced LLM Servingcited by 13
  6. 2024Designing Large Foundation Models for Efficient Training and Inference: A Surveycited by 7
  7. 2024Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradientcited by 6
  8. 2025DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentrationcited by 6
  9. 2025SparseSSM: Efficient Selective Structured State Space Models Can Be Pruned in One-Shotcited by 5
  10. 2024Deploying Foundation Model Powered Agent Services: A Surveycited by 4
  11. 2025RAP: Runtime Adaptive Pruning for LLM Inferencecited by 3
  12. 2024A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Modelscited by 2
4 more
  1. 2025AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Designcited by 1
  2. 2024Mixed Sparsity Training: Achieving 4× FLOP Reduction for Transformer Pretrainingcited by 0
  3. 2025CUT: Pruning Pre-Trained Multi-Task Models into Compact Models for Edge Devicescited by 0
  4. 2026Event-Driven Language Models with Sparse Neural Activity for Neuromorphic Hardwarecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
  2. 2024Aggressive Post-Training Compression on Extremely Large Language Models
  3. 2025The Unseen Frontier: Pushing the Limits of LLM Sparsity with Surrogate-Free ADMM
  4. 2026SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
  5. 2026Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compression
  6. 2025CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Models
  7. 2024Pruning Foundation Models for High Accuracy without Retraining
  8. 2024Pruning Large Language Models with Semi-Structural Adaptive Sparse Training