mlsanityData to 7 Oct 2026

RLTF: Reinforcement Learning from Unit Test Feedback

arXiv 2307.04349 ↗First posted 10 Jul 2023cs.AI cs.CL cs.LGcited by 110

Not on the map. Why?

Cited by 20 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The goal of program synthesis, or code generation, is to generate executable code based on given descriptions. Recently, there has been an increasing number of studies employing reinforcement learning (RL) to improve the performance of large language models (LLMs) for code. However, current representative works either rely solely on offline frameworks, limiting the exploration of new sample spaces, or fall short in the utilization of unit test signals, not accounting for specific error locations within the code. To address these issues, we propose RLTF, i.e., Reinforcement Learning from Unit Test Feedback, a novel online RL framework with unit test feedback of multi-granularity for refining code LLMs. Our approach generates data in real-time during training and simultaneously utilizes fine-grained feedback signals to guide the model towards producing higher-quality code. Extensive experiments show that RLTF achieves state-of-the-art performance on the APPS and the MBPP benchmarks. Our code is available at: https://github.com/Zyq-scut/RLTF.

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Builds on 29 of its 34 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

29 of 34 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  9. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  10. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  11. 2021Measuring Coding Challenge Competence With APPScited by 879
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
17 more
  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  2. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  3. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  4. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  5. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  6. 2022Self-critiquing models for assisting human evaluatorscited by 372
  7. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  8. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  9. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  10. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  11. 2023SantaCoder: don't reach for the stars!cited by 169
  12. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 144
  13. 2022Large Language Models Meet NL2Code: A Surveycited by 135
  14. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 95
  15. 2021Online and Offline Reinforcement Learning by Planning with a Learned Modelcited by 90
  16. 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 47
  17. 2022Repairing Bugs in Python Assignments Using Large Language Modelscited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 29; 2025: 30; 2026: 46.

110 papers.

Sort:

By area:

  1. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  2. 2024A Survey on Large Language Models for Code Generationcited by 646
  3. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  4. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  5. 2024ODIN: Disentangled Reward Mitigates Hacking in RLHFcited by 133
  6. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  7. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  8. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  9. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  10. 2024RaFe: Ranking Feedback Improves Query Rewriting for RAGcited by 47
  11. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  12. 2026Code as Agent Harnesscited by 40
38 more
  1. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generationcited by 34
  2. 2024Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learningcited by 32
  3. 2024ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedbackcited by 28
  4. 2024Process Supervision-Guided Policy Optimization for Code Generationcited by 25
  5. 2024IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generatorscited by 21
  6. 2025Process-Supervised Reinforcement Learning for Code Generationcited by 21
  7. 2025Outcome-based Reinforcement Learning to Predict the Futurecited by 21
  8. 2024Learning Code Preference via Synthetic Evolutioncited by 19
  9. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  10. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  11. 2023B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesiscited by 16
  12. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  13. 2024Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learningcited by 15
  14. 2024MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actionscited by 15
  15. 2026Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysiscited by 15
  16. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  17. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  18. 2025ReachAgent: Enhancing Mobile Agent via Page Reaching and Operationcited by 13
  19. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  20. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  21. 2025Exploring Code Language Models for Automated HLS-based Hardware Generation: Benchmark, Infrastructure and Analysiscited by 10
  22. 2024ITERTL: An Iterative Framework for Fine-tuning LLMs for RTL Code Generationcited by 9
  23. 2024Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Codecited by 9
  24. 2024Aligning CodeLLMs with Direct Preference Optimizationcited by 9
  25. 2024Multi-Programming Language Sandbox for LLMscited by 9
  26. 2024DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMscited by 9
  27. 2025Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learningcited by 9
  28. 2026GameGen-Verifier: Parallel Keypoint-Based Verification for LLM-Generated Games via Runtime State Injectioncited by 9
  29. 2024PLUM: Improving Code LMs with Execution-Guided On-Policy Preference Learning Driven By Synthetic Test Casescited by 8
  30. 2024FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization systemcited by 8
  31. 2025Scoring Verifiers: Evaluating Synthetic Verification for Code and Reasoningcited by 8
  32. 2025A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Modelscited by 7
  33. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RLcited by 7
  34. 2024Flaming-hot Initiation with Regular Execution Sampling for Large Language Modelscited by 6
  35. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  36. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 6
  37. 2026Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVRcited by 6
  38. 2025Supervised Optimism Correction: Be Confident When LLMs Are Surecited by 5

50 most cited of 110 All 110 (JSON)