Product-Aware Deterministic Rounding for Quantized Matrix Multiplication
Not on the map. Why?
Abstract (arXiv)
Scalar rounding decisions interact through matrix multiplication. We study deterministic product-aware rounding after scales, clipping bounds, and grids are fixed, with each active scalar choosing between adjacent levels. For dynamic activation rounding, null-space reduction preserves the relaxed product while leaving at most r fractional decisions, where r is the rank of the active gap-weighted weight block. Conditional-expectation completion gives a deterministic polynomial-time algorithm with squared product error at most OPTdyn+rnumax2/4, where OPTdyn is the best admissible error and numax is the largest row norm of that block. For reusable static weights, the exact expected product-loss metric is the uncentered input second moment with fixed output bias; free bias recalibration yields the centered covariance. Exact optimization is NP-hard even at rank one. In balanced blocks with K=1024 and r=16, conditional- expectation completion attains a dither-normalized median error of 0.010, compared with 0.899 for round-to-nearest. Clipping-aware initialization reduces median normalized error by a factor of 43.4 at ten-percent clipping. Held-out Digits experiments show that retaining the input mean or correcting the output bias improves median product error over round-to-nearest in all four tested bit-width and calibration-size settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 32 references that are in the map. First posted on arXiv 11 Sep 2026.
Builds on
11 of 32 references.
References · 11
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
- 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 20
- 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantizationcited by 15
- 2024Optimal Quantization for Matrix Multiplicationcited by 14
- 2017The Gram-Schmidt Walk: A Cure for the Banaszczyk Bluescited by 10
- 2025The Lattice Geometry of Neural Network Quantization -- A Short Equivalence Proof of GPTQ and Babai's Algorithmcited by 9
- 2025DiscQuant: A Quantization Method for Neural Networks Inspired by Discrepancy Theorycited by 3
- 2026Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transitioncited by 2
- 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 2
- 2026A Nuclear-Norm Lower Bound for Dithered Scalar Quantization of Matrix Productscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Rounding in Preconditioner Space: Redesigning 4-bit AdamW Optimizer-State Quantization
- 2023Stella Nera: A Differentiable Maddness-Based Hardware Accelerator for Efficient Approximate Matrix Multiplication
- 2026Stochastic Rounding Increases Small Singular Values
- 2026Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement
- 2026Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication
- 2026KBBQ: A Predictive Noise Law and the Limits of Spectrum Flattening in FP4 Quantization
- 2025The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference
- 2026BCJR-QAT: A Differentiable Relaxation of Trellis-Coded Weight Quantization