mlsanityData to 14 Oct 2026

AWP: Activation-Aware Weight Pruning and Quantization with Projected Gradient Descent

arXiv 2506.10205 ↗First posted 11 Jun 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

To address the enormous size of Large Language Models (LLMs), model compression methods, such as quantization and pruning, are often deployed, especially on edge devices. In this work, we focus on layer-wise post-training quantization and pruning. Drawing connections between activation-aware weight pruning and sparse approximation problems, and motivated by the success of Iterative Hard Thresholding (IHT), we propose a unified method for Activation-aware Weight pruning and quantization via Projected gradient descent (AWP). Our experiments demonstrate that AWP outperforms state-of-the-art LLM pruning and quantization methods. Theoretical convergence guarantees of the proposed method for pruning are also provided.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

26 of 40 references.

References · 26

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  7. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  8. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  10. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  11. 2021A White Paper on Neural Network Quantizationcited by 472
  12. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
14 more
  1. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  2. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  3. 2023A Survey on Model Compression for Large Language Modelscited by 269
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  5. 2023Efficient Large Language Models: A Surveycited by 186
  6. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 80
  7. 2014On Iterative Hard Thresholding Methods for High-dimensional M-Estimationcited by 53
  8. 2024Q-VLM: Post-training Quantization for Large Vision-Language Modelscited by 47
  9. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33
  10. 2020A Greedy Algorithm for Quantizing Neural Networkscited by 22
  11. 2018Between hard and soft thresholding: optimal iterative thresholding algorithmscited by 15
  12. 20241-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUscited by 12
  13. 2016Average-case Hardness of RIP Certificationcited by 10
  14. 2024Unified Stochastic Framework for Neural Network Quantization and Pruningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Flycited by 1
  2. 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
  3. 2026From Attention Sensitivity to Layer Role: Revisiting Mixed-Precision Quantization of Transformerscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Amber Pruner: Leveraging N:M Activation Sparsity for Efficient Prefill in Large Language Models
  2. 2025Choose Your Model Size: Any Compression of Large Language Models Without Re-Computation
  3. 2024Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix
  4. 2024Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradient
  5. 2023Compresso: Structured Pruning with Collaborative Prompting Learns Compact Large Language Models
  6. 2026Structural Pruning of Large Vision Language Models: A Comprehensive Study on Pruning Dynamics, Recovery, and Data Efficiency
  7. 2024Aggressive Post-Training Compression on Extremely Large Language Models
  8. 2025E3-Pruner: Towards Efficient, Economical, and Effective Layer Pruning for Large Language Models