# Model-Preserving Adaptive Rounding

arXiv: https://arxiv.org/abs/2505.22988 · HTML: https://mlsanity.com/paper/2505.22988/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 28 of 28)

- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Fast Inference from Transformers via Speculative Decoding](https://mlsanity.com/paper/2211.17192/) (2022)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [A Simple and Effective Pruning Approach for Large Language Models](https://mlsanity.com/paper/2306.11695/) (2023)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [LLM-QAT: Data-Free Quantization Aware Training for Large Language Models](https://mlsanity.com/paper/2305.17888/) (2023)
- [SqueezeLLM: Dense-and-Sparse Quantization](https://mlsanity.com/paper/2306.07629/) (2023)
- [Extreme Compression of Large Language Models via Additive Quantization](https://mlsanity.com/paper/2401.06118/) (2024)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis](https://mlsanity.com/paper/1806.03884/) (2018)
- [Adam-mini: Use Fewer Learning Rates To Gain More](https://mlsanity.com/paper/2406.16793/) (2024)
- [Overcoming Oscillations in Quantization-Aware Training](https://mlsanity.com/paper/2203.11086/) (2022)
- [QTIP: Quantization with Trellises and Incoherence Processing](https://mlsanity.com/paper/2406.11235/) (2024)
- [PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression](https://mlsanity.com/paper/2405.14852/) (2024)
- [A New Perspective on Shampoo's Preconditioner](https://mlsanity.com/paper/2406.17748/) (2024)
- [Training LLMs with MXFP4](https://mlsanity.com/paper/2502.20586/) (2025)
- [CBQ: Cross-Block Quantization for Large Language Models](https://mlsanity.com/paper/2312.07950/) (2023)
- [GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance](https://mlsanity.com/paper/2505.07004/) (2025)
- [NestQuant: Nested Lattice Quantization for Matrix Products and LLMs](https://mlsanity.com/paper/2502.09720/) (2025)
- [Hierarchical Roofline Performance Analysis for Deep Learning Applications](https://mlsanity.com/paper/2009.05257/) (2020)
- [Model Preserving Compression for Neural Networks](https://mlsanity.com/paper/2108.00065/) (2021)
- [DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theory](https://mlsanity.com/paper/2501.06417/) (2025)
- [A Two Pronged Progress in Structured Dense Matrix Multiplication](https://mlsanity.com/paper/1611.01569/) (2016)

## Built on by (most cited first, 23 of 23)

- [Training Dynamics Impact Post-Training Quantization Robustness](https://mlsanity.com/paper/2510.06213/) (2025)
- [WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization](https://mlsanity.com/paper/2603.04956/) (2026)
- [Leech Lattice Vector Quantization for Efficient LLM Compression](https://mlsanity.com/paper/2603.11021/) (2026)
- [CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization](https://mlsanity.com/paper/2602.05902/) (2026)
- [Statistically-Lossless Quantization of Large Language Models](https://mlsanity.com/paper/2605.02404/) (2026)
- [HARP: Hadamard-Preconditioned Adaptive Rotation Processor for Extreme LLM Quantization](https://mlsanity.com/paper/2605.29843/) (2026)
- [Optimal Brain Decomposition for Accurate LLM Low-Rank Approximation](https://mlsanity.com/paper/2604.00821/) (2026)
- [High-Rate Quantized Matrix Multiplication II](https://mlsanity.com/paper/2605.13768/) (2026)
- [LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization](https://mlsanity.com/paper/2606.10531/) (2026)
- [KronQ: LLM Quantization via Kronecker-Factored Hessian](https://mlsanity.com/paper/2607.07964/) (2026)
- [GPTQ-2D: Cubic-Time Two-Sided Adaptive Rounding](https://mlsanity.com/paper/2607.27042/) (2026)
- [MGVQ: Synergizing Multi-dimensional Sensitivity-Aware and Gradient-Hessian Fusion for Vector Quantization](https://mlsanity.com/paper/2605.24019/) (2026)
- [FastKron: Efficient Quantization with Kronecker-Factored Hessians](https://mlsanity.com/paper/2608.06291/) (2026)
- [QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction](https://mlsanity.com/paper/2608.13966/) (2026)
- [Why Does Post-Training Quantization Work?](https://mlsanity.com/paper/2609.11716/) (2026)
- [Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models](https://mlsanity.com/paper/2607.14630/) (2026)
- [Pricing the Risk of Runtime Compression: Anytime-Valid Admission and a Served-Output Law for Compressed Serving State](https://mlsanity.com/paper/2608.15810/) (2026)
- [WaterKron and FlipFlop Hessian: Information-Theoretically Grounded Quantization with Kronecker-factored Hessians](https://mlsanity.com/paper/2609.14706/) (2026)
- [Predicting Quantization Price for Selecting PTQ Configurations Before Deployment](https://mlsanity.com/paper/2609.28270/) (2026)
- [G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation](https://mlsanity.com/paper/2609.31009/) (2026)
- [XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization](https://mlsanity.com/paper/2610.00432/) (2026)
- [Revisiting Error Feedback for Model Quantization](https://mlsanity.com/paper/2610.18139/) (2026)
- [Two For The Price Of One: Curvature-Aware Pruning Preserves Influence Functions Too](https://mlsanity.com/paper/2610.18516/) (2026)

## Abstract (arXiv)

The goal of quantization is to produce a compressed model whose output distribution is as close to the original model's as possible. To do this tractably, most quantization algorithms minimize the immediate activation error of each layer as a proxy for the end-to-end error. However, this ignores the effect of future layers, making it a poor proxy. In this work, we introduce Yet Another Quantization Algorithm (YAQA), an adaptive rounding algorithm that directly considers the error at the network's output. YAQA introduces a series of theoretical results that culminate in the first end-to-end error bounds for quantization algorithms. First, we characterize the convergence time of adaptive rounding algorithms via the structure of their Hessian approximations. We then show that the end-to-end error can be bounded by the approximation's cosine similarity to the true Hessian. This admits a natural Kronecker-factored approximation with corresponding near-optimal Hessian sketches. YAQA is provably better than GPTQ/LDLQ and empirically reduces the error by ≈ 30% over these methods. YAQA even achieves a lower error than quantization aware training. This translates to state of the art performance on downstream tasks, all while adding no inference overhead.