Data to 14 Oct 2026

GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibration

arXiv 2504.02692 ↗First posted 3 Apr 2025cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

We introduce GPTAQ, a novel finetuning-free quantization method for compressing large-scale transformer architectures. Unlike the previous GPTQ method, which independently calibrates each layer, we always match the quantized layer's output to the exact output in the full-precision model, resulting in a scheme that we call asymmetric calibration. Such a scheme can effectively reduce the quantization error accumulated in previous layers. We analyze this problem using optimal brain compression to derive a close-formed solution. The new solution explicitly minimizes the quantization error as well as the accumulated asymmetry error. Furthermore, we utilize various techniques to parallelize the solution calculation, including channel parallelization, neuron decomposition, and Cholesky reformulation for matrix fusion. As a result, GPTAQ is easy to implement, simply using 20 more lines of code than GPTQ but improving its performance under low-bit quantization. Remarkably, on a single GPU, we quantize a 405B language transformer as well as EVA-02, the rank first vision transformer that achieves 90% pretraining Imagenet accuracy. Code is available at Github.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Builds on 40 of its 47 references that are in the map. First posted on arXiv 3 Apr 2025.

Builds on

40 of 47 references.

References · 40

2017Attention Is All You Needcited by 60,361
2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
2020Training data-efficient image transformers & distillation through attentioncited by 4,586
2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386

28 more

2021Scaling Vision Transformerscited by 999
2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
2024SpinQuant: LLM quantization with learned rotationscited by 471
2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
2023EVA-02: A Visual Representation for Neon Genesiscited by 381
2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
2021Post-Training Quantization for Vision Transformercited by 278
2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 92
2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 64
2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 48.

60 papers.

Sort:

By area:

2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 28
2025Sensitivity-LoRA: Low-Load Sensitivity-Based Fine-Tuning for Large Language Modelscited by 14
2025Quant-dLLM: Post-Training Extreme Low-Bit Quantization for Diffusion Large Language Modelscited by 12
2026What Makes Low-Bit Quantization-Aware Training Work for Reasoning LLMs? A Systematic Studycited by 10
2025First-Order Error Matters: Accurate Compensation for Quantized Large Language Modelscited by 8
2025SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Modelscited by 8
2026KBVQ-MoE: KLT-guided SVD with Bias-Corrected Vector Quantization for MoE Large Language Modelscited by 8
2026Rethinking Residual Errors in Compensation-based LLM Quantizationcited by 7
2025DuoGPT: Training-free Dual Sparsity through Activation-aware Pruning in LLMscited by 6
2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 6
2025LoaQ: Layer-wise Output Approximation Quantizationcited by 5

38 more

2025PT2-LLM: Post-Training Ternarization for Large Language Modelscited by 4
2026Optimal Brain Decomposition for Accurate LLM Low-Rank Approximationcited by 4
2025A Comprehensive Evaluation on Quantization Techniques for Large Language Modelscited by 3
2026LoPRo: Enhancing Low-Rank Quantization via Permuted Block-Wise Rotationcited by 3
2026TurboBoA: Faster and Exact Attention-aware Quantization without Backpropagationcited by 3
2026A Guide to Large Language Models in Modeling and Simulation: From Core Techniques to Critical Challengescited by 3
2026TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantizationcited by 3
2026Optimal Post-Training Quantization Scales and Where to Find Themcited by 3
2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 3
2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMscited by 2
2026EdgeRazor: A Lightweight Framework for Large Language Models via Mixed-Precision Quantization-Aware Distillationcited by 2
2026Saliency-Aware Regularized Quantization Calibration for Large Language Modelscited by 2
2026Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Scorecited by 2
2026Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMscited by 2
2026H-Scale: Hessian-Guided Scale Refinement for NVFP4 Sub-Byte LLM Inferencecited by 2
2025NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMscited by 1
2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPUcited by 1
2026TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tilingcited by 1
2026CAR-SAM: Cross-Attention Reconstruction for Post-Training Quantization of the Segment Anything Modelcited by 1
2026MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantizationcited by 1
2026Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inferencecited by 1
2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 1
2026ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantizationcited by 1
2026Why Does Post-Training Quantization Work?cited by 1
2025Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Modelscited by 0
2026RPIQ: Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization for Visually Impaired Assistancecited by 0
2026On the Importance of a Multi-Scale Calibration for Quantizationcited by 0
2026Online Learning for Multi-Layer Hierarchical Inference under Partial and Policy-Dependent Feedbackcited by 0
2026MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantizationcited by 0
2026SPEAR: A System for Post-Quantization Error-Adaptive Recovery Enabling Efficient Low-Bit LLM Servingcited by 0
2026An Empirical Study of openPangu Quantization on Ascend NPUscited by 0
2026LogbQuant: Quantizing Language Models in Logarithmic Spacecited by 0
2026One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantizationcited by 0
2026SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantizationcited by 0
2026SandwichQuant: Which Parameters Matter Before and After Quantization?cited by 0
2026REAL-Q: E2E LLM Quantization via Dynamic Gradient Descentcited by 0
2026Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residualscited by 0
2026FoldQuantVLA: Native Low-Bit Quantization of Vision-Language-Action Models via Consistent Foldingcited by 0

50 most cited of 60 All 60 (JSON)

Similar, not cited

Close in wording, no citation either way