mlsanityData to 14 Oct 2026

PoTPTQ: A Two-step Power-of-Two Post-training for LLMs

arXiv 2507.11959 ↗First posted 16 Jul 2025cs.CL cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable performance across various natural language processing (NLP) tasks. However, their deployment is challenging due to the substantial computational resources required. Power-of-two (PoT) quantization is a general tool to counteract this difficulty. Albeit previous works on PoT quantization can be efficiently dequantized on CPUs using fixed-point addition, it showed less effectiveness on GPUs. The reason is entanglement of the sign bit and sequential bit manipulations needed for dequantization. We propose a novel POT quantization framework for LLM weights that (i) outperforms state-of-the-art accuracy in extremely low-precision number formats, and (ii) enables faster inference through more efficient dequantization. To maintain the accuracy of the quantized model, we introduce a two-step post-training algorithm: (i) initialize the quantization scales with a robust starting point, and (ii) refine these scales using a minimal calibration set. The performance of our PoT post-training algorithm surpasses the current state-of-the-art in integer quantization, particularly at low precisions such as 2- and 3-bit formats. Our PoT quantization accelerates the dequantization step required for the floating point inference and leads to 3.67× speed up on a NVIDIA V100, and 1.63× on a NVIDIA RTX 4090, compared to uniform integer dequantization.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

21 of 29 references.

References · 21

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  8. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  9. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  10. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  11. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  12. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
9 more
  1. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  2. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  3. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  4. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 54
  5. 2020ShiftAddNet: A Hardware-Inspired Deep Networkcited by 52
  6. 2022Power-of-Two Quantization for Low Bitwidth and Hardware Compliant Neural Networkscited by 26
  7. 2022RAPQ: Rescuing Accuracy for Power-of-Two Low-bit Post-training Quantizationcited by 15
  8. 2021S3: Sign-Sparse-Shift Reparametrization for Effective Training of Low-bit Shift Networkscited by 7
  9. 2022DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantizationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026HoloQ-VLA: Uniform W4A4 Quantization of Vision-Language-Action Modelscited by 2
  2. 2026PoTAcc: A Pipeline for End-to-End Acceleration of Power-of-Two Quantized DNNscited by 1
  3. 2025Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs
  2. 2026Power-of-Two Quantization-Aware-Training (PoT-QAT) in Large Language Models (LLMs)
  3. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
  4. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
  5. 2025SplitQuantV2: Enhancing Low-Bit Quantization of LLMs Without GPUs
  6. 2025PT2-LLM: Post-Training Ternarization for Large Language Models
  7. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
  8. 2024Post Training Quantization of Large Language Models with Microscaling Formats