mlsanityData to 13 Oct 2026

JARQ: Joint Alternating Refinement for Quantization

arXiv 2609.38599 ↗First posted 29 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Group-wise post-training quantizers for large language models round weights onto a grid that is not refit to the resulting integer codes. We show that this leaves accuracy on the table: the best grid depends on the codes, input correlations couple the errors of different groups, and useful code changes often involve many codes at once. We propose JARQ , a plug-in refinement that starts from any group-wise quantizer and alternates a joint least-squares fit of all group scales with bounded Babai proposals that move many codes of a group together on the current grid. The problem is a bilinear box-constrained mixed-integer least-squares problem; the solver is backpropagation-free, does not increase the layer-wise objective under exact scale solves, and keeps the host's bit width, groups, zero points, and inference cost. Across Llama-2, Llama-3, and Qwen models with RTN, GPTQ, OmniQuant, and AWQ hosts, JARQ lowers perplexity in 90 of 96 comparisons, cuts three-bit RTN perplexity by up to 36%, raises mean multiple-choice accuracy in 23 of 24 configurations, and improves QEP, QuaRot, and OJBKQ outputs, at under a minute per 7B block.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 32 of its 37 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

32 of 37 references.

References · 32

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,549
  2. 2024The Llama 3 Herd of Modelscited by 14,486
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  5. 2025Qwen3 Technical Reportcited by 8,510
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,270
  8. 2024Qwen2.5 Technical Reportcited by 4,840
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,798
  10. 2023Let's Verify Step by Stepcited by 3,672
  11. 2016Pointer Sentinel Mixture Modelscited by 3,047
  12. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,379
20 more
  1. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,163
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,884
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,365
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,272
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 617
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 614
  7. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 472
  8. 2024SpinQuant: LLM quantization with learned rotationscited by 456
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 413
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 406
  11. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 367
  12. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 318
  13. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 49
  14. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 38
  15. 2018Alternating Multi-bit Quantization for Recurrent Neural Networkscited by 31
  16. 2023QuantEase: Optimization-based Quantization for Language Modelscited by 28
  17. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 26
  18. 2025The Geometry of LLM Quantization: GPTQ as Babai's Nearest Plane Algorithmcited by 19
  19. 2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 6
  20. 2026ADMM-Q: An Improved Hessian-based Weight Quantizer for Post-Training Quantization of Large Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Two-Stage Grid Optimization for Group-wise Quantization of LLMs
  2. 2026SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization
  3. 2026ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantization
  4. 2026Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
  5. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
  6. 2026BPDQ: Bit-Plane Decomposition Quantization on a Variable Grid for Large Language Models
  7. 2023Norm Tweaking: High-performance Low-bit Quantization of Large Language Models
  8. 2026REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent