mlsanityData to 7 Oct 2026

AFPQ: Asymmetric Floating Point Quantization for LLMs

arXiv 2311.01792 ↗First posted 3 Nov 2023cs.CL cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) show great performance in various tasks, but face deployment challenges from limited memory capacity and bandwidth. Low-bit weight quantization can save memory and accelerate inference. Although floating-point (FP) formats show good performance in LLM quantization, they tend to perform poorly with small group sizes or sub-4 bits. We find the reason is that the absence of asymmetry in previous FP quantization makes it unsuitable for handling asymmetric value distribution of LLM weight tensors. In this work, we propose asymmetric FP quantization (AFPQ), which sets separate scales for positive and negative values. Our method leads to large accuracy improvements and can be easily plugged into other quantization methods, including GPTQ and AWQ, for better performance. Besides, no additional storage is needed compared with asymmetric integer (INT) quantization. The code is available at https://github.com/zhangsichengsjtu/AFPQ.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

20 of 24 references.

References · 20

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  12. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
8 more
  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  3. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  4. 20218-bit Optimizers via Block-wise Quantizationcited by 395
  5. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  6. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  7. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44
  8. 2023Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Modelscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 3.

7 papers.

  1. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 17
  2. 2024HQ-DiT: Efficient Diffusion Transformer with FP4 Hybrid Quantizationcited by 15
  3. 2025Rotate, Clip, and Partition: Towards W2A4KV4 Quantization by Integrating Rotation and Learnable Non-uniform Quantizercited by 3
  4. 2025FPQVAR: Floating Point Quantization for Visual Autoregressive Model with FPGA Hardware Co-designcited by 0
  5. 2026Benford's Law as a Distributional Prior for Post-Training Quantization of Large Language Modelscited by 0
  6. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 0
  7. 2026GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inferencecited by 0