mlsanityData to 7 Oct 2026

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization

arXiv 2311.05161 ↗First posted 9 Nov 2023cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are proficient in natural language processing tasks, but their deployment is often restricted by extensive parameter sizes and computational demands. This paper focuses on post-training quantization (PTQ) in LLMs, specifically 4-bit weight and 8-bit activation (W4A8) quantization, to enhance computational efficiency -- a topic less explored compared to weight-only quantization. We present two innovative techniques: activation-quantization-aware scaling (AQAS) and sequence-length-aware calibration (SLAC) to enhance PTQ by considering the combined effects on weights and activations and aligning calibration sequence lengths to target tasks. Moreover, we introduce dINT, a hybrid data format combining integer and denormal representations, to address the underflow issue in W4A8 quantization, where small values are rounded to zero. Through rigorous evaluations of LLMs, including OPT and LLaMA, we demonstrate that our techniques significantly boost task accuracies to levels comparable with full-precision models. By developing arithmetic units compatible with dINT, we further confirm that our methods yield a 2× hardware efficiency improvement compared to 8-bit integer MAC unit.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

21 of 35 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  11. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
9 more
  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  3. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  4. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  5. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  6. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  7. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  8. 2021Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formatscited by 26
  9. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 11; 2025: 6; 2026: 2.

20 papers.

Sort:

By area:

  1. 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 51
  2. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  3. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  4. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 17
  5. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 13
  6. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matricescited by 11
  7. 2024Post Training Quantization of Large Language Models with Microscaling Formatscited by 10
  8. 2024Improving Conversational Abilities of Quantized Large Language Models via Direct Preference Alignmentcited by 9
  9. 2025QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Modelscited by 9
  10. 2025Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimizationcited by 9
  11. 2024RankAdaptor: Hierarchical Rank Allocation for Efficient Fine-Tuning Pruned LLMs via Performance Modelcited by 7
  12. 2024DecDEC: A Systems Approach to Advancing Low-Bit LLM Quantizationcited by 7
8 more
  1. 2024A Structure-Aware Framework for Learning Device Placements on Computation Graphscited by 3
  2. 2024QPruner: Probabilistic Decision Quantization for Structured Pruning in Large Language Modelscited by 2
  3. 2025Energy-Efficient and Dequantization-Free Q-LLMs: A Spiking Neural Network Approach to Salient Value Mitigationcited by 2
  4. 2025Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2cited by 1
  5. 2024AutoMixQ: Self-Adjusting Quantization for High Performance Memory-Efficient Fine-Tuningcited by 0
  6. 2025Q&C: When Quantization Meets Cache in Efficient Image Generationcited by 0
  7. 2026Efficient Reasoning on the Edgecited by 0
  8. 2026P4Q: Co-designing Token Pruning and Quantization for Vision-Language Model Accelerationcited by 0