mlsanityData to 14 Oct 2026

Model-Preserving Adaptive Rounding

arXiv 2505.22988 ↗First posted 29 May 2025cs.LG cs.AIcited by 23

Not on the map. Why?

Abstract (arXiv)

The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by ≈ 30% over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 28 of its 45 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

28 of 45 references.

References · 28

  1. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  5. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2019Learned Step Size Quantizationcited by 628
  8. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  9. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 426
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  11. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  12. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
16 more
  1. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  2. 2019Limitations of the Empirical Fisher Approximation for Natural Gradient Descentcited by 217
  3. 2018Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasiscited by 146
  4. 2024Adam-mini: Use Fewer Learning Rates To Gain Morecited by 137
  5. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  6. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  7. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  8. 2024A New Perspective on Shampoo's Preconditionercited by 59
  9. 2025Training LLMs with MXFP4cited by 37
  10. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  11. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  12. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 23
  13. 2020Hierarchical Roofline Performance Analysis for Deep Learning Applicationscited by 11
  14. 2021Model Preserving Compression for Neural Networkscited by 9
  15. 2025DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theorycited by 3
  16. 2016A Two Pronged Progress in Structured Dense Matrix Multiplicationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 22.

23 papers.

Sort:

By area:

  1. 2025Training Dynamics Impact Post-Training Quantization Robustnesscited by 15
  2. 2026WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantizationcited by 7
  3. 2026Leech Lattice Vector Quantization for Efficient LLM Compressioncited by 7
  4. 2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 6
  5. 2026Statistically-Lossless Quantization of Large Language Modelscited by 6
  6. 2026HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantizationcited by 5
  7. 2026Optimal Brain Decomposition for Accurate LLM Low-Rank Approximationcited by 4
  8. 2026High-Rate Quantized Matrix Multiplication IIcited by 4
  9. 2026LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantizationcited by 3
  10. 2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 3
  11. 2026GPTQ-2D: Cubic-Time Two-Sided Adaptive Roundingcited by 2
  12. 2026MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantizationcited by 1
11 more
  1. 2026FastKron: Efficient Quantization with Kronecker-Factored Hessianscited by 1
  2. 2026QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstructioncited by 1
  3. 2026Why Does Post-Training Quantization Work?cited by 1
  4. 2026Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Modelscited by 0
  5. 2026Pricing the Risk of Runtime Compression: Anytime-Valid Admission and a Served-Output Law for Compressed Serving Statecited by 0
  6. 2026WaterKron and FlipFlop Hessian: Information-Theoretically Grounded Quantization with Kronecker-factored Hessianscited by 0
  7. 2026Predicting Quantization Price for Selecting PTQ Configurations Before Deploymentcited by 0
  8. 2026G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensationcited by 0
  9. 2026XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantizationcited by 0
  10. 2026Revisiting Error Feedback for Model Quantizationcited by 0
  11. 2026Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Toocited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Error-aware Quantization through Noise Tempering
  2. 2026HeRo-Q: A General Framework for Stable Low Bit Quantization via Hessian Conditioning
  3. 2026Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantization
  4. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization
  5. 2022Efficient Adaptive Activation Rounding for Post-Training Quantization
  6. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm
  7. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
  8. 2025CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding