mlsanityData to 14 Oct 2026

Revisiting Error Feedback for Model Quantization

arXiv 2610.18139 ↗First posted 13 Oct 2026cs.LG eess.SPcited by 0

Not on the map. Why?

Abstract (arXiv)

We study causal error-feedback coefficients as design variables for model quantization. Modeling instantaneous quantization errors as independent, zero-mean variables with a common fixed variance, we formulated a metric-dependent trace problem for designing feedback coefficients. The formulation resembles fixed-order GPTQ/LDLQ feedback and supports Jacobian-aware designs that account for downstream error propagation. For a positive-definite metric boldsymbolH, we prove that the joint rotation-feedback trace minimum is ndet(boldsymbolH)1/n and present an attaining pair using the geometric mean decomposition. Experiments on synthetic networks, trained MLPs, and language models show benefits from downstream-sensitive feedback, but also reveal that trace optimality need not improve finite-level performance. Small QAT experiments further illustrate direct loss-driven feedback tuning with fixed or trainable network weights.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 32 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

20 of 32 references.

References · 20

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  8. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  9. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  11. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  12. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
8 more
  1. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  2. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
  3. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 32
  4. 2025Model-Preserving Adaptive Roundingcited by 23
  5. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 20
  6. 2025The Lattice Geometry of Neural Network Quantization -- A Short Equivalence Proof of GPTQ and Babai's Algorithmcited by 9
  7. 2019One-Bit Sigma-Delta MIMO Precodingcited by 3
  8. 2023Spatial Sigma-Delta Modulation for Coarsely Quantized Massive MIMO Downlink: Flexible Designs by Convex Optimizationcited by 1

Built on by

Not cited yet.