# IRCoCo: Immediate Rewards-Guided Deep Reinforcement Learning for Code Completion

arXiv: https://arxiv.org/abs/2401.16637 · HTML: https://mlsanity.com/paper/2401.16637/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jan 2024.

## Built on by (most cited first, 13 of 13)

- [CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code](https://mlsanity.com/paper/2404.15639/) (2024)
- [What to Retrieve for Effective Retrieval-Augmented Code Generation? An Empirical Study and Beyond](https://mlsanity.com/paper/2503.20589/) (2025)
- [Enhancing Code Generation Performance of Smaller Models by Distilling the Reasoning Ability of LLMs](https://mlsanity.com/paper/2403.13271/) (2024)
- [Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learning](https://mlsanity.com/paper/2407.18271/) (2024)
- [Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedback](https://mlsanity.com/paper/2504.15804/) (2025)
- [Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach](https://mlsanity.com/paper/2404.14296/) (2024)
- [CODESYNC: Synchronizing Large Language Models with Dynamic Code Evolution at Scale](https://mlsanity.com/paper/2502.16645/) (2025)
- [Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detection](https://mlsanity.com/paper/2506.18245/) (2025)
- [StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning](https://mlsanity.com/paper/2605.11922/) (2026)
- [Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning](https://mlsanity.com/paper/2502.05739/) (2025)
- [Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization](https://mlsanity.com/paper/2605.11974/) (2026)
- [Router-Suggest: Dynamic Routing for Multimodal Auto-Completion in Visually-Grounded Dialogs](https://mlsanity.com/paper/2601.05851/) (2026)
- [PYTHALAB-MERA: Validation-Grounded Memory, Retrieval, and Acceptance Control for Frozen-LLM Coding Agents](https://mlsanity.com/paper/2605.08468/) (2026)

## Abstract (arXiv)

Code completion aims to enhance programming productivity by predicting potential code based on the current programming context. Recently, pretrained language models (LMs) have become prominent in this field. Various approaches have been proposed to fine-tune LMs using supervised fine-tuning (SFT) techniques for code completion. However, the inherent exposure bias of these models can cause errors to accumulate early in the sequence completion, leading to even more errors in subsequent completions. To address this problem, deep reinforcement learning (DRL) is an alternative technique for fine-tuning LMs for code completion, which can improve the generalization capabilities and overall performance. Nevertheless, integrating DRL-based strategies into code completion faces two major challenges: 1) The dynamic nature of the code context requires the completion model to quickly adapt to changes, which poses difficulties for conventional DRL strategies that focus on delayed rewarding of the final code state. 2) It is difficult to evaluate the correctness of partial code, thus the reward redistribution-based strategies cannot be adapted to code completion. To tackle these challenges, we propose IRCoCo, a code completion-specific DRL-based fine-tuning framework. This framework is designed to provide immediate rewards as feedback for detecting dynamic context changes arising from continuous edits during code completion. With the aid of immediate feedback, the fine-tuned LM can gain a more precise understanding of the current context, thereby enabling effective adjustment of the LM and optimizing code completion in a more refined manner. Experimental results demonstrate that fine-tuning pretrained LMs with IRCoCo leads to significant improvements in the code completion task, outperforming both SFT-based and other DRL-based baselines.