# GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration

arXiv: https://arxiv.org/abs/2504.02692 · HTML: https://mlsanity.com/paper/2504.02692/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 47 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [EVA-02: A Visual Representation for Neon Genesis](https://mlsanity.com/paper/2303.11331/) (2023)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning](https://mlsanity.com/paper/2208.11580/) (2022)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://mlsanity.com/paper/1705.07565/) (2017)
- [BitNet: Scaling 1-bit Transformers for Large Language Models](https://mlsanity.com/paper/2310.11453/) (2023)
- [Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models](https://mlsanity.com/paper/2209.13325/) (2022)
- [QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization](https://mlsanity.com/paper/2203.05740/) (2022)

## Built on by (most cited first, 30 of 60)

- [Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization](https://mlsanity.com/paper/2504.09629/) (2025)
- [Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization](https://mlsanity.com/paper/2505.11695/) (2025)
- [Sensitivity-LoRA: Low-Load Sensitivity-Based Fine-Tuning for Large Language Models](https://mlsanity.com/paper/2509.09119/) (2025)
- [Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Models](https://mlsanity.com/paper/2510.03274/) (2025)
- [What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Study](https://mlsanity.com/paper/2601.14888/) (2026)
- [First-Order Error Matters: Accurate Compensation for Quantized Large Language Models](https://mlsanity.com/paper/2507.11017/) (2025)
- [SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models](https://mlsanity.com/paper/2508.03351/) (2025)
- [KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Models](https://mlsanity.com/paper/2602.11184/) (2026)
- [Rethinking Residual Errors in Compensation-based LLM Quantization](https://mlsanity.com/paper/2604.07955/) (2026)
- [DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMs](https://mlsanity.com/paper/2506.20194/) (2025)
- [CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization](https://mlsanity.com/paper/2602.05902/) (2026)
- [LoaQ: Layer-wise Output Approximation Quantization](https://mlsanity.com/paper/2509.06297/) (2025)
- [PT2-LLM: Post-Training Ternarization for Large Language Models](https://mlsanity.com/paper/2510.03267/) (2025)
- [Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation](https://mlsanity.com/paper/2604.00821/) (2026)
- [A Comprehensive Evaluation on Quantization Techniques for Large Language Models](https://mlsanity.com/paper/2507.17417/) (2025)
- [LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotation](https://mlsanity.com/paper/2601.19675/) (2026)
- [TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagation](https://mlsanity.com/paper/2602.04929/) (2026)
- [A Guide to Large Language Models in Modeling and Simulation: From Core Techniques to Critical Challenges](https://mlsanity.com/paper/2602.05883/) (2026)
- [TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization](https://mlsanity.com/paper/2605.19561/) (2026)
- [Optimal Post-Training Quantization Scales and Where to Find Them](https://mlsanity.com/paper/2606.10890/) (2026)
- [KronQ: LLM Quantization via Kronecker-Factored Hessian](https://mlsanity.com/paper/2607.07964/) (2026)
- [D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs](https://mlsanity.com/paper/2602.02546/) (2026)
- [EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillation](https://mlsanity.com/paper/2605.04062/) (2026)
- [Saliency-Aware Regularized Quantization Calibration for Large Language Models](https://mlsanity.com/paper/2605.05693/) (2026)
- [Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score](https://mlsanity.com/paper/2606.13300/) (2026)
- [Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs](https://mlsanity.com/paper/2608.08506/) (2026)
- [H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inference](https://mlsanity.com/paper/2608.28113/) (2026)
- [NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMs](https://mlsanity.com/paper/2505.17595/) (2025)
- [A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU](https://mlsanity.com/paper/2602.17693/) (2026)
- [TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling](https://mlsanity.com/paper/2605.09281/) (2026)

## Abstract (arXiv)

We introduce GPTAQ, a novel finetuning-free quantization method for compressing large-scale transformer architectures. Unlike the previous GPTQ method, which independently calibrates each layer, we always match the quantized layer's output to the exact output in the full-precision model, resulting in a scheme that we call asymmetric calibration. Such a scheme can effectively reduce the quantization error accumulated in previous layers. We analyze this problem using optimal brain compression to derive a close-formed solution. The new solution explicitly minimizes the quantization error as well as the accumulated asymmetry error. Furthermore, we utilize various techniques to parallelize the solution calculation, including channel parallelization, neuron decomposition, and Cholesky reformulation for matrix fusion. As a result, GPTAQ is easy to implement, simply using 20 more lines of code than GPTQ but improving its performance under low-bit quantization. Remarkably, on a single GPU, we quantize a 405B language transformer as well as EVA-02, the rank first vision transformer that achieves 90% pretraining Imagenet accuracy. Code is available at Github.