Fine-Tuning Low-Bit Models with Gradient in Quantized Code Space
Not on the map. Why?
Abstract (arXiv)
Fine-tuning Low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practically important as it reduces memory and inference cost for storage and deployment. Under this constraint, adaptation becomes an optimization problem over quantization codes and scales. Existing continuous low-bit training is efficient, but it can be distorted by straight through estimation error or by post-quantize gap; discrete search is deployment-faithful, but it is often too inefficient under a finite training budget. We propose code surrogate gradient as the first order signal in deployable code space to acceleate optimization, and performing guided search to preserve deployment faithfulness. Experiments across arithmetic reasoning, instruction following, and structured language understanding show that GradCodes consistently improves fine-tuning low-bit models across different quantization datatypes. Code is provided at https://github.com/ovo67/GradCodes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
18 of 21 references.
References · 18
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
- 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
- 2023LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Modelscited by 196
- 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
- 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
- 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
- 2022A Langevin-like Sampler for Discrete Distributionscited by 44
- 2022Discrete Langevin Sampler via Wasserstein Gradient Flowcited by 24
- 2024L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Modelscited by 15
- 2024LoQT: Low-Rank Adapters for Quantized Pretrainingcited by 12
6 more
- 2025QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Modelscited by 11
- 2023Efficient Informed Proposals for Discrete Distributions via Newton's Series Approximationcited by 7
- 2022Gradient-Guided Importance Sampling for Learning Binary Energy-Based Modelscited by 6
- 2025Reheated Gradient-based Discrete Sampling for Combinatorial Optimizationcited by 6
- 2025Fine-tuning Quantized Neural Networks with Zeroth-order Optimizationcited by 6
- 2026Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Costcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
- 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths
- 2025End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost
- 2024Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
- 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Models
- 2025QLESS: A Quantized Approach for Data Valuation and Selection in Large Language Model Fine-Tuning
- 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment
- 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization