mlsanityData to 7 Oct 2026

JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates

arXiv 2605.25469 ↗First posted 25 May 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Quantization-aware training (QAT) is widely deployed but typically relies on the Straight-Through Estimator (STE), which passes gradients through non-differentiable quantizers by fiat. This often makes training brittle near bin boundaries and weakly aligned with the actual behavior of the low-precision model. We introduce JacQuant, a QAT framework that learns a lightweight surrogate of the model's local sensitivity to parameter changes and uses it to stabilize and accelerate training within standard variance-reduced optimizers. The surrogate is inexpensive (diagonal or block-diagonal), data-driven, and compatible with common weight and activation quantizers. On code-preserving training phases, we prove convergence for non-convex objectives and obtain linear rates under a PL condition, and we relate the learned sensitivity to end-to-end output fidelity via a simple calibration argument. Across LLM benchmarks at ≤ 2 bits, JacQuant consistently reaches higher accuracy than STE-based QAT, and the runtime analyses on various models show that the added cost remains negligible under practical group sizes. The method is drop-in and requires no changes to the forward quantizers; our empirical claims are scoped to ultra-low-bit LLM QAT.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 39 of its 45 references that are in the map. First posted on arXiv 25 May 2026.

Builds on

39 of 45 references.

References · 39

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  7. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  8. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  11. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  12. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
27 more
  1. 2019Learned Step Size Quantizationcited by 604
  2. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  3. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  4. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 526
  5. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  6. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  7. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  8. 2025Muon is Scalable for LLM Trainingcited by 381
  9. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
  10. 2019Momentum-Based Variance Reduction in Non-Convex SGDcited by 302
  11. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 225
  12. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  13. 2017SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradientcited by 218
  14. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 180
  15. 2017Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representationscited by 149
  16. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
  17. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  18. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  19. 2024Restructuring Vector Quantization with the Rotation Trickcited by 49
  20. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  21. 2015Variance Reduced Stochastic Gradient Descent with Neighborscited by 39
  22. 2018Stochastic Quasi-Gradient Methods: Variance Reduction via Jacobian Sketchingcited by 34
  23. 2022Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learningcited by 33
  24. 2021ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computationcited by 28
  25. 2024Adaptive Variance Reduction for Stochastic Optimization under Weaker Assumptionscited by 5
  26. 2025Improving the Straight-Through Estimator with Zeroth-Order Informationcited by 5
  27. 2025End-to-End On-Device Quantization-Aware Training for LLMs at Inference Costcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Quantization-Aware Pre-Training with Constrained Empirical Weight Distributioncited by 0
  2. 2026Few Bits, One Law: Toward W2A4KV2cited by 0