# High-Rate Quantized Matrix Multiplication II

arXiv: https://arxiv.org/abs/2605.13768 · HTML: https://mlsanity.com/paper/2605.13768/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 16 of 16)

- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [QTIP: Quantization with Trellises and Incoherence Processing](https://mlsanity.com/paper/2406.11235/) (2024)
- [Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantization](https://mlsanity.com/paper/2505.11695/) (2025)
- [NestQuant: Nested Lattice Quantization for Matrix Products and LLMs](https://mlsanity.com/paper/2502.09720/) (2025)
- [Model-Preserving Adaptive Rounding](https://mlsanity.com/paper/2505.22988/) (2025)
- [The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithm](https://mlsanity.com/paper/2507.18553/) (2025)
- [Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos](https://mlsanity.com/paper/2508.04853/) (2025)
- [Optimal Quantization for Matrix Multiplication](https://mlsanity.com/paper/2410.13780/) (2024)
- [The Lattice Geometry of Neural Network Quantization -- A Short Equivalence Proof of GPTQ and Babai's Algorithm](https://mlsanity.com/paper/2508.01077/) (2025)
- [High-Rate Quantized Matrix Multiplication I](https://mlsanity.com/paper/2601.17187/) (2026)
- [WaterSIC: Information-Theoretically (Near) Optimal Linear Layer Quantization](https://mlsanity.com/paper/2603.04956/) (2026)
- [Price of metric universality in vector quantization is at most 0.11 bit](https://mlsanity.com/paper/2602.05790/) (2026)

## Built on by (most cited first, 4 of 4)

- [Contraction-Gauge Preconditioning for Quantized Matrix Multiplication](https://mlsanity.com/paper/2607.18745/) (2026)
- [A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products](https://mlsanity.com/paper/2609.05641/) (2026)
- [Transforms for LLM Quantization: The Great Inversion and Format Co-Design](https://mlsanity.com/paper/2608.25188/) (2026)
- [OSFP4: Joint Optimization of Diagonal Smoothing and Block Scales for NVFP4 Quantization](https://mlsanity.com/paper/2610.08231/) (2026)

## Abstract (arXiv)

This is the second part of the work investigating quantized matrix multiplication (MatMul). In part I we considered the case of calibration-free quantization, whereas here we discuss the setting where covariance matrix SigmaX of the columns of the second factor is available. This setting arises in the ubiquitous task of weight-only post-training quantization of LLMs. Weight-only quantization is related to the problem of weighted mean squared error (WMSE) source coding, whose classical (reverse) waterfilling solution dictates how one should distribute rate between coordinates of the vector. We show how waterfilling can be used to improve practical LLM quantization algorithms (GPTQ), which at present allocate rate equally. A recent scheme (known as ``WaterSIC'') that only uses scalar INT quantizers is analyzed and its high-rate performance is shown to be (a) basis free (i.e., characterized by the determinant of SigmaX and, thus, unlike existing schemes, is immune to applying random rotations); and (b) within a multiplicative factor of frac2π e12 (or 0.25 bit/entry) of the information-theoretic distortion limit. GPTQ's performance, in turn, is affected by the choice of basis, but for a random rotation and actual SigmaX from Llama-3-8B we find it to be within 0.1 bit (depending on the layer type) of WaterSIC, suggesting that GPTQ with random rotation is also near optimal, at least in the high-rate regime.