mlsanityData to 7 Oct 2026

R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization

arXiv 2511.21736 ↗First posted 21 Nov 2025cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The rapid progress of Large Language Models (LLMs) has brought substantial computational and memory demands, spurring the adoption of low-bit quantization. While 8-bit and 4-bit formats have become prevalent, extending quantization to 2 bits remains challenging due to severe accuracy degradation. To address this, we propose Residual Refinement Quantization (R2Q)-a novel 2-bit quantization framework that decomposes the process into two sequential 1-bit sub-quantizations, forming an adaptive quantization lattice. Extensive evaluations on Llama, OPT, and Qwen across diverse benchmarks-covering question answering, commonsense reasoning, and language modeling-demonstrate that R2Q consistently outperforms existing 2-bit quantization methods in both fine-grained and coarse-grained settings. By refining quantization through a residual learning mechanism, R2Q enhances performance, improves training stability, and accelerates convergence under extreme compression. Furthermore, its modular design enables seamless integration with existing quantization-aware training (QAT) frameworks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 47 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

38 of 47 references.

References · 38

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2016Pointer Sentinel Mixture Modelscited by 2,992
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
26 more
  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  5. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  7. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  11. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  12. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 289
  13. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 227
  14. 2016Ternary Weight Networkscited by 195
  15. 2022FP8 Formats for Deep Learningcited by 187
  16. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 178
  17. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 175
  18. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 116
  19. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 80
  20. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 77
  21. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
  22. 2023Training Transformers with 4-bit Integerscited by 58
  23. 2024TernaryLLM: Ternarized Large Language Modelcited by 15
  24. 2024ViT-1.58b: Mobile Vision Transformers in the 1-bit Eracited by 7
  25. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 7
  26. 2025iFairy: the First 2-bit Complex LLM with All Parameters in ±1, ± icited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
  2. 2025PTQTP: Post-Training Quantization to Trit-Planes for Large Language Models
  3. 2026UniSVQ: 2-bit Unified Scalar-Vector Quantization
  4. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Models
  5. 2026All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs
  6. 2025LittleBit: Ultra Low-Bit Quantization via Latent Factorization
  7. 2026SliderQuant: Accurate Post-Training Quantization for LLMs
  8. 2025MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration