mlsanityData to 14 Oct 2026

A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Products

arXiv 2609.05641 ↗First posted 4 Sep 2026cs.IT cs.LG cs.NAcited by 1

Not on the map. Why?

Abstract (arXiv)

We consider the problem of minimizing error in quantized matrix multiplication C=AB. Scalar quantization of the factors introduces rounding errors whose scale depends on the maximum absolute entries -- the ranges -- of their rows and columns. These ranges determine the quantization grid steps. To reduce the error, we optimize over product-preserving transformations that alter the factor ranges and grid steps without changing C. Specifically, we seek the smallest leading expected squared error over invertible inner changes of basis and orthogonal outer rotations. Under independent, zero-mean subtractive dither noise on an unbounded lattice, we prove the output-only bound Erm lead ≥ (cA+cB)/K Vert ABVert_*2, where K is the inner dimension, cA and cB are normalized noise variances, and Vert ABVert_* is the nuclear norm. The bound is tight: an SVD-aligned Hadamard construction attains the infimum whenever a Hadamard matrix of order K exists, including every power of two, while an SVD-aligned DCT construction is within a factor of two for every K. Without outer rotations, Gram-matrix balancing minimizes factorization energy, and finite-set flattening achieves the bound within Clog(K(m+n)). For power-of-two K, conditional expectations deterministically select the Hadamard signs in O((m+n)K2) exact-real operations. Synthetic experiments verify both constructions and illustrate the tradeoff between regularization and conditioning. These results characterize the full-gauge optimum and quantify the cost of preserving row and column indices.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 8 of its 18 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

8 of 18 references.

References · 8

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  2. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  3. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  4. 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantizationcited by 15
  5. 2024Optimal Quantization for Matrix Multiplicationcited by 14
  6. 2026High-Rate Quantized Matrix Multiplication Icited by 7
  7. 2026High-Rate Quantized Matrix Multiplication IIcited by 4
  8. 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition
  2. 2026KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
  3. 2026Provable Quantization with Randomized Hadamard Transform
  4. 2025Dimensionally-Efficient Transmission and Storage of Unitary Matrices ↗
  5. 2024Rejection-Sampled Universal Quantization for Smaller Quantization Errors
  6. 2026A Note on Scaling in Randomly Rotated Quantization and Its Connection to the CDEF +1 Pythagorean Relation
  7. 2026Research Report on Noise-Shaped One-Bit Coefficients in Discrete Polynomial Fourier Extension
  8. 2023Matrix Completion from One-Bit Dither Samples