# CYCLE: Learning to Self-Refine the Code Generation

arXiv: https://arxiv.org/abs/2403.18746 · HTML: https://mlsanity.com/paper/2403.18746/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 55 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 30 of 38)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [A Systematic Evaluation of Large Language Models of Code](https://mlsanity.com/paper/2202.13169/) (2022)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://mlsanity.com/paper/2110.07178/) (2021)
- [Efficient Training of Language Models to Fill in the Middle](https://mlsanity.com/paper/2207.14255/) (2022)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)

## Built on by (most cited first, 30 of 42)

- [Beyond Functional Correctness: Exploring Hallucinations in LLM-Generated Code](https://mlsanity.com/paper/2404.00971/) (2024)
- [From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future](https://mlsanity.com/paper/2408.02479/) (2024)
- [Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement](https://mlsanity.com/paper/2411.00622/) (2024)
- [Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration](https://mlsanity.com/paper/2406.01422/) (2024)
- [Code Repair with LLMs gives an Exploration-Exploitation Tradeoff](https://mlsanity.com/paper/2405.17503/) (2024)
- [From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging](https://mlsanity.com/paper/2410.01215/) (2024)
- [LeDex: Training LLMs to Better Self-Debug and Explain Code](https://mlsanity.com/paper/2405.18649/) (2024)
- [AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMs](https://mlsanity.com/paper/2507.05687/) (2025)
- [VulnBot: Autonomous Penetration Testing for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2501.13411/) (2025)
- [QualityFlow: An Agentic Workflow for Program Synthesis Controlled by LLM Quality Checks](https://mlsanity.com/paper/2501.17167/) (2025)
- [Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks](https://mlsanity.com/paper/2508.13143/) (2025)
- [debug-gym: A Text-Based Environment for Interactive Debugging](https://mlsanity.com/paper/2503.21557/) (2025)
- [Are LLMs Correctly Integrated into Software Systems?](https://mlsanity.com/paper/2407.05138/) (2024)
- [Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewards](https://mlsanity.com/paper/2505.02686/) (2025)
- [RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance](https://mlsanity.com/paper/2410.01242/) (2024)
- [Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment](https://mlsanity.com/paper/2502.07803/) (2025)
- [API-guided Dataset Synthesis to Finetune Large Code Models](https://mlsanity.com/paper/2408.08343/) (2024)
- [Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://mlsanity.com/paper/2504.14640/) (2025)
- [Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency](https://mlsanity.com/paper/2505.02133/) (2025)
- [Validating Network Protocol Parsers with Traceable RFC Document Interpretation](https://mlsanity.com/paper/2504.18050/) (2025)
- [QCoder Benchmark: Bridging Language Generation and Quantum Hardware through Simulator-Based Feedback](https://mlsanity.com/paper/2510.26101/) (2025)
- [Differentiation-Based Extraction of Proprietary Data from Fine-Tuned LLMs](https://mlsanity.com/paper/2506.17353/) (2025)
- [Large Language Models for Validating Network Protocol Parsers](https://mlsanity.com/paper/2504.13515/) (2025)
- [How Many Tries Does It Take? Iterative Self-Repair in LLM Code Generation Across Model Scales and Benchmarks](https://mlsanity.com/paper/2604.10508/) (2026)
- [SeCodePLT: A Unified Platform for Evaluating the Security of Code GenAI](https://mlsanity.com/paper/2410.11096/) (2024)
- [Falsification, Not Exposure: An Internally Preregistered Placebo-Controlled Decomposition of Self-Repair Feedback in Frozen Small Code Models](https://mlsanity.com/paper/2606.31511/) (2026)
- [Mastering the Craft of Data Synthesis for CodeLLMs](https://mlsanity.com/paper/2411.00005/) (2024)
- [Leveraging Metamemory Agent for Enhanced Data-Free Code Generation in Large Language Models](https://mlsanity.com/paper/2501.07892/) (2025)
- [Teaching LLM to Reason: Reinforcement Learning from Algorithmic Problems without Code](https://mlsanity.com/paper/2507.07498/) (2025)
- [SDA: Steering-Driven Distribution Alignment for Open LLMs without Fine-Tuning](https://mlsanity.com/paper/2511.16324/) (2025)

## Abstract (arXiv)

Pre-trained code language models have achieved promising performance in code generation and improved the programming efficiency of human developers. However, their self-refinement capability is typically overlooked by the existing evaluations of code LMs, which focus only on the accuracy of the one-time prediction. For the cases when code LMs fail to implement the correct program, developers actually find it hard to debug and fix the faulty prediction since it is not written by the developers themselves. Unfortunately, our study reveals that code LMs cannot efficiently self-refine their faulty generations as well. In this paper, we propose CYCLE framework, learning to self-refine the faulty generation according to the available feedback, such as the execution results reported by the test suites. We evaluate CYCLE on three popular code generation benchmarks, HumanEval, MBPP, and APPS. The results reveal that CYCLE successfully maintains, sometimes improves, the quality of one-time code generation, while significantly improving the self-refinement capability of code LMs. We implement four variants of CYCLE with varied numbers of parameters across 350M, 1B, 2B, and 3B, and the experiments show that CYCLE consistently boosts the code generation performance, by up to 63.5%, across benchmarks and varied model sizes. We also notice that CYCLE outperforms code LMs that have 3× more parameters in self-refinement.