mlsanityData to 7 Oct 2026

Wanda++: Pruning Large Language Models via Regional Gradients

arXiv 2503.04992 ↗First posted 6 Mar 2025cs.LG cs.AI cs.CLcited by 26

On the map: Efficient LLM inference and serving → Early exit and edge inference

Abstract (arXiv)

Large Language Models (LLMs) pruning seeks to remove unimportant weights for inference speedup with minimal accuracy impact. However, existing methods often suffer from accuracy degradation without full-model sparsity-aware fine-tuning. This paper presents Wanda++, a novel pruning framework that outperforms the state-of-the-art methods by utilizing decoder-block-level regional gradients. Specifically, Wanda++ improves the pruning score with regional gradients for the first time and proposes an efficient regional optimization method to minimize pruning-induced output discrepancies between the dense and sparse decoder output. Notably, Wanda++ improves perplexity by up to 32% over Wanda in the language modeling task and generalizes effectively to downstream tasks. Moreover, despite updating weights with regional optimization, Wanda++ remains orthogonal to sparsity-aware fine-tuning, further reducing perplexity with LoRA in great extend. Our approach is lightweight, pruning a 7B LLaMA model in under 10 minutes on a single H100 GPU.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Early exit and edge inference, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 45 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

27 of 45 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  7. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  10. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  11. 2016An overview of gradient descent optimization algorithmscited by 1,021
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
15 more
  1. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  2. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  3. 2019The State of Sparsity in Deep Neural Networkscited by 471
  4. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  5. 2022Language model compression with weighted low-rank factorizationcited by 192
  6. 2022FP8 Quantization: The Power of the Exponentcited by 87
  7. 2021Revisiting Locally Supervised Learning: an Alternative to End-to-end Trainingcited by 71
  8. 2022Structural Pruning via Latency-Saliency Knapsackcited by 40
  9. 2024Pruner-Zero: Evolving Symbolic Pruning Metric from scratch for Large Language Modelscited by 40
  10. 2023Beyond Size: How Gradients Shape Pruning Decisions in Large Language Modelscited by 30
  11. 2023HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformerscited by 25
  12. 2024Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Taskscited by 22
  13. 2022BEBERT: Efficient and Robust Binary Ensemble BERTcited by 13
  14. 2023How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspectivecited by 9
  15. 2023Efficient N:M Sparse DNN Training Using Algorithm, Architecture, and Dataflow Co-Designcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 18.

26 papers.

Sort:

By area:

  1. 2025SepPrune: Structured Pruning for Efficient Deep Speech Separationcited by 31
  2. 2025Block-Sparse Global Attention for Efficient Multi-View Geometry Transformerscited by 22
  3. 2026Garbage Attention in Large Language Models: BOS Sink Heads and Sink-aware Pruningcited by 7
  4. 2026On State Reduction in Linear Attentioncited by 6
  5. 2025FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compressioncited by 5
  6. 2025From Local to Global: Revisiting Structured Pruning Paradigms for Large Language Modelscited by 5
  7. 2025ACE: Exploring Activation Cosine Similarity and Variance for Accurate and Calibration-Efficient LLM Pruningcited by 2
  8. 2026LLMs can Compress LLMs: Adaptive Pruning by Agentscited by 2
  9. 2026GRASPrune: Global Gating for Budgeted Structured Pruning of Large Language Modelscited by 2
  10. 2026Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AIcited by 2
  11. 2025Frustratingly Easy Task-aware Pruning for Large Language Modelscited by 1
  12. 2025TwIST: Rigging the Lottery in Transformers with Independent Subnetwork Trainingcited by 1
14 more
  1. 2026Sink-Aware Pruning for Diffusion Language Modelscited by 1
  2. 2026Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantizationcited by 1
  3. 2025Can abstract concepts from LLM improve SLM performance?cited by 0
  4. 2026TraceNAS: Zero-shot LLM Pruning via Gradient Trace Correlationcited by 0
  5. 2026UniComp: A Unified Evaluation of Large Language Model Compression via Pruning, Quantization, and Distillationcited by 0
  6. 2026GRAIL: Post-hoc Compensation by Linear Reconstruction for Compressed Networkscited by 0
  7. 2026Alternating Gradient Flow Utility: A Unified Metric for Structural Pruning and Dynamic Routing in Deep Networkscited by 0
  8. 2026Structural Sensitivity in Compressed Transformers: Relative Error Propagation and Layer Removalcited by 0
  9. 2026Small LLMs: Pruning vs. Training from Scratchcited by 0
  10. 2026Weight-Adjusted Gradients Reveal Parameter Importance and Failure Modes in LLMscited by 0
  11. 2026TriSP: Tri-Signal Structured Pruning for Large Language Modelscited by 0
  12. 2026At-the-Roofline Sparse Tensor Contractions on Vector Processors for Transformer Inferencecited by 0
  13. 2026The Asymmetric Harms of LLM Compressioncited by 0
  14. 2026Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exitcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Pruning Laws for Large Language Models
  2. 2025Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Information
  3. 2025SPAP: Structured Pruning via Alternating Optimization and Penalty Methods
  4. 2026MI-PRUN: Optimize Large Language Model Pruning via Mutual Information
  5. 2025Automatic Pruning Discovery for Large Language Models
  6. 2023Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models
  7. 2024Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
  8. 2025MaskPrune: Mask-based LLM Pruning for Layer-wise Uniform Structures