mlsanityData to 7 Oct 2026

ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization

arXiv 2411.07762 ↗First posted 12 Nov 2024cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Quantization stands as a pivotal technique for large language model (LLM) serving, yet it poses significant challenges particularly in achieving effective low-bit quantization. The limited numerical mapping makes the quantized model produce a non-trivial error, bringing out intolerable performance degration. This paper is anchored in the basic idea of model compression objectives, and delves into the layer-wise error distribution of LLMs during post-training quantization. Subsequently, we introduce ASER, an algorithm consisting of (1) Error Reconstruction: low-rank compensation for quantization error with LoRA-style matrices constructed by whitening SVD; (2) Activation Smoothing: outlier extraction to gain smooth activation and better error compensation. ASER is capable of quantizing typical LLMs to low-bit ones, particularly preserving accuracy even in W4A8 per-channel setup. Experimental results show that ASER is competitive among the state-of-the-art quantization algorithms, showing potential to activation quantization, with minor overhead.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 12 Nov 2024.

Builds on

25 of 33 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  11. 2016Pointer Sentinel Mixture Modelscited by 2,992
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
13 more
  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  5. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  6. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  7. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  8. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
  9. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  10. 2020Contrastive Distillation on Intermediate Representations for Language Model Compressioncited by 57
  11. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  12. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMscited by 17
  13. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLMcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 63
  2. 2025Layer-wise dynamic rank for compressing large language modelscited by 6
  3. 2026MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Modelscited by 5
  4. 2026Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantizationcited by 3
  5. 2026GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMscited by 0
  6. 2026HCInfer: An Efficient Inference System via Error Compensation for Resource-Constrained Devicescited by 0
  7. 2026SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Servingcited by 0
  8. 2026Predicting Quantization Price for Selecting PTQ Configurations Before Deploymentcited by 0