Quantized Precoding for Maximizing Sum Rate in MU-MIMO Systems with Constrained Fronthaul

This paper studies a downlink multi-user multiple-input multiple-output (MU-MIMO) system, where the precoding matrix is computed at a baseband unit (BBU) and then transmitted to the remote antenna array over a limited-capacity digital fronthaul. The limited bit resolution of the fronthaul introduces quantization effects that are explicitly modeled. We propose a novel sum rate maximization framework that directly incorporates the quantizer's constraints into the precoding design. The resulting maximization problem, a non-convex mixed-integer program, is addressed using a new iterative algorithm inspired by the weighted minimum mean square error (WMMSE) methodology. The precoding optimization subproblem is reformulated as an integer least-squares problem and solved using a novel sphere decoding (SD) algorithm. Additionally, a low-complexity expectation propagation (EP)-based method is introduced to enable the practical implementation of quantized precoding in MU-massive MIMO (MU-mMIMO) systems. Furthermore, numerical evaluations demonstrate that the proposed precoding schemes outperform conventional approaches that optimize infinite-resolution precoding followed by element-wise quantization. We also propose a heuristic quantization-aware precoding method with comparable complexity to the baseline but superior performance. In particular, the EP-based approach offers near-optimal performance with substantial complexity reduction, making it well-suited for real-time MU-mMIMO applications.