Optimal Scalar Quantization for Matrix Multiplication: Closed-Form Density and Phase Transition

We study entrywise scalar quantization of two matrices prior to multiplication. Given A∈ Rm× k and B∈ Rk× n, we quantize entries of A and B independently using scalar quantizers with KX and KY levels per entry, and form widehat C=widehat A,widehat B. The objective is to minimize the matrix multiplication mean-squared error (MSE) E[|AB-widehat Awidehat B|F2] under a pair-i.i.d. inner-product model. In the high-resolution regime KX,KY→∞, we derive a sharp K-2 asymptotic expansion for E, identify the exact optimal leading constants, and characterize asymptotically optimal quantization center densities in terms of conditional second moments. We then specialize to correlated Gaussian multiplicative pairs, obtaining a closed-form optimal point density [ λ^star(u) propto exp!left(-fracu26right)bigl((1-ρ2)+ρ2u2bigr)1/3, qquad u=fracxσX, ] with the same form for y/σY, and prove a correlation-driven phase transition: the density is unimodal at the origin for |ρ|≤ 1/√3 and becomes bimodal for |ρ|>1/√3 with peaks at upeak=±√3-1/ρ2. We show our method's applicability in synthetic experiments such as matrix multiplication quantization and least squares optimization, as well as quantization of large language model key and query activations.