mlsanityData to 14 Oct 2026

RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning

arXiv 2410.02089 ↗First posted 2 Oct 2024cs.CL cs.AIcited by 151

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large language models (LLMs) deployed as agents solve user-specified tasks over multiple steps while keeping the required manual engagement to a minimum. Crucially, such LLMs need to ground their generations in any feedback obtained to reliably achieve the desired outcomes. We propose an end-to-end reinforcement learning method for teaching models to leverage execution feedback in the realm of code synthesis, where state-of-the-art LLMs struggle to improve code iteratively compared to independent sampling. We benchmark on competitive programming tasks, where we achieve new state-of-the art results with both small (8B parameters) and large (70B) models while reducing the amount of samples required by an order of magnitude. Our analysis of inference-time behavior demonstrates that our method produces LLMs that effectively leverage automatic feedback over multiple steps.

Key facts. Cited by 151 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 46 of its 52 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

46 of 52 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solving18 May
  2. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
  3. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback2 Feb
  4. 2023Teaching Large Language Models to Self-Debug11 Apr
  5. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,916
29 more
  1. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  4. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  5. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  6. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  7. 2023StarCoder: may the source be with you!cited by 799
  8. 2023Textbooks Are All You Needcited by 569
  9. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
  10. 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
  11. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  12. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  13. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  14. 2024AutoCodeRover: Autonomous Program Improvementcited by 278
  15. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  16. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 228
  17. 2024AI Agents That Mattercited by 219
  18. 2023Is Self-Repair a Silver Bullet for Code Generation?cited by 197
  19. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  20. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learningcited by 158
  21. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  22. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  23. 2024Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineeringcited by 103
  24. 2023Execution-based Code Generation using Deep Reinforcement Learningcited by 99
  25. 2020PyMT5: multi-mode translation of natural language and Python code with transformerscited by 80
  26. 2023Improving Code Generation by Training with Natural Language Feedbackcited by 76
  27. 2024TestGenEval: A Real World Unit Test Generation and Test Completion Benchmarkcited by 52
  28. 2020On Learning Meaningful Assert Statements for Unit Test Casescited by 42
  29. 2023B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesiscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 69; 2026: 78.

151 papers.

Sort:

By area:

  1. 2025Gemma 3 Technical Reportcited by 1,824
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  3. 2026Reinforcement Learning via Self-Distillationcited by 332
  4. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  5. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 205
  6. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  7. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 156
  8. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thoughtcited by 109
  9. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 93
  10. 2025CWM: An Open-Weights LLM for Research on Code Generation with World Modelscited by 93
  11. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 87
  12. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 83
38 more
  1. 2025Reinforcing General Reasoning without Verifierscited by 77
  2. 2025AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-benchcited by 71
  3. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Codecited by 61
  4. 2025Reinforcement Learning from Human Feedbackcited by 58
  5. 2025Optimizing Language Models for Inference Time Objectives using Reinforcement Learningcited by 53
  6. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  7. 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Taskscited by 49
  8. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 47
  9. 2026Code as Agent Harnesscited by 47
  10. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 46
  11. 2025Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Usecited by 45
  12. 2025LlamaRL: A Distributed Asynchronous Reinforcement Learning Framework for Efficient Large-scale LLM Trainingcited by 42
  13. 2025Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGIcited by 36
  14. 2025Robotouille: An Asynchronous Planning Benchmark for LLM Agentscited by 32
  15. 2024Taking AI Welfare Seriouslycited by 31
  16. 2025ARE: Scaling Up Agent Environments and Evaluationscited by 30
  17. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 29
  18. 2025Beyond Verifiable Rewards: Scaling Reinforcement Learning for Language Models to Unverifiable Datacited by 29
  19. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 28
  20. 2026Towards Execution-Grounded Automated AI Researchcited by 28
  21. 2025Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applicationscited by 26
  22. 2024ProSec: Fortifying Code LLMs with Proactive Security Alignmentcited by 25
  23. 2024PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedbackcited by 24
  24. 2025Learning to Orchestrate Agents in Natural Language with the Conductorcited by 23
  25. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  26. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  27. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimizationcited by 18
  28. 2025Adaptive Guidance Accelerates Reinforcement Learning of Reasoning Modelscited by 18
  29. 2025HelpSteer3: Human-Annotated Feedback and Edit Data to Empower Inference-Time Scaling in Open-Ended General-Domain Taskscited by 17
  30. 2025VisCoder: Fine-Tuning LLMs for Executable Python Visualization Code Generationcited by 17
  31. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  32. 2025Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQLcited by 14
  33. 2025World Modelling Improves Language Model Agentscited by 14
  34. 2025Soft Policy Optimization: Online Off-Policy RL for Sequence Modelscited by 12
  35. 2025RL-finetuning LLMs from on- and off-policy data with a single algorithmcited by 11
  36. 2025From Reasoning to Code: GRPO Optimization for Underrepresented Languagescited by 11
  37. 2025Scaling Graph Chain-of-Thought Reasoning: A Multi-Agent Framework with Efficient LLM Servingcited by 10
  38. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skillscited by 10

50 most cited of 151 All 151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026MaxCode: A Max-Reward Reinforcement Learning Framework for Automated Code Optimization
  2. 2024CHAI for LLMs: Improving Code-Mixed Translation in Large Language Models through Reinforcement Learning with AI Feedback
  3. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
  4. 2024Process Supervision-Guided Policy Optimization for Code Generation
  5. 2024Applying RLAIF for Code Generation with API-usage in Lightweight LLMs
  6. 2026Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs
  7. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence
  8. 2025ACE-RLHF: Automated Code Evaluation and Socratic Feedback Generation Tool using Large Language Models and Reinforcement Learning with Human Feedback