mlsanityData to 7 Oct 2026

StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback

arXiv 2402.01391 ↗First posted 2 Feb 2024cs.SE cs.CLcited by 74

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 31 of its 42 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

31 of 42 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Competition-Level Code Generation with AlphaCode8 Feb

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2023Let's Verify Step by Stepcited by 3,598
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
18 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 879
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  4. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  5. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  6. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  7. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  8. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  9. 2017Reverse Curriculum Generation for Reinforcement Learningcited by 185
  10. 2023SantaCoder: don't reach for the stars!cited by 169
  11. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  12. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoningcited by 96
  13. 2023Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedbackcited by 78
  14. 2018Learning Montezuma's Revenge from a Single Demonstrationcited by 76
  15. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 57
  16. 2022PanGu-Coder: Program Synthesis with Function-Level Language Modelingcited by 47
  17. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 46
  18. 2016Playing Atari Games with Deep Reinforcement Learning and Human Checkpoint Replaycited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 34; 2026: 20.

74 papers.

Sort:

By area:

  1. 2024A Survey on Large Language Models for Code Generationcited by 646
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2024OpenCodeInterpreter: Integrating Code Generation with Execution and Refinementcited by 242
  4. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  5. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  6. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  7. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 83
  8. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 82
  9. 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 68
  10. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  11. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  12. 2024The Larger the Better? Improved LLM Code-Generation via Budget Reallocationcited by 41
38 more
  1. 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learningcited by 34
  2. 2024CodeS: Natural Language to Code Repository via Multi-Layer Sketchcited by 28
  3. 2024Process Supervision-Guided Policy Optimization for Code Generationcited by 25
  4. 2024PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedbackcited by 22
  5. 2025Process-Supervised Reinforcement Learning for Code Generationcited by 21
  6. 2025ChemDFM-R: A Chemical Reasoning LLM Enhanced with Atomized Chemical Knowledgecited by 20
  7. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  8. 2024Towards a Unified View of Preference Learning for Large Language Models: A Surveycited by 17
  9. 2024Large Language Model for Verilog Generation with Code-Structure-Guided Reinforcement Learningcited by 15
  10. 2024ReflectionCoder: Learning from Reflection Sequence for Enhanced One-off Code Generationcited by 14
  11. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  12. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimizationcited by 14
  13. 2024RLSF: Fine-tuning LLMs via Symbolic Feedbackcited by 13
  14. 2024Meta-Designing Quantum Experiments with Language Modelscited by 12
  15. 2025A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Codecited by 12
  16. 2025Training Language Models to Generate Quality Code with Program Analysis Feedbackcited by 11
  17. 2025From Intent to Execution: Multimodal Chain-of-Thought Reinforcement Learning for Precise CAD Code Generationcited by 10
  18. 2024Multi-Programming Language Sandbox for LLMscited by 9
  19. 2024DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMscited by 9
  20. 2024FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization systemcited by 8
  21. 2025Repair-R1: Better Test Before Repaircited by 6
  22. 2026On Information Self-Locking in Reinforcement Learning for Active Reasoning of LLM agentscited by 6
  23. 2025SuperCoder: Assembly Program Superoptimization with Large Language Modelscited by 5
  24. 2025Improving LLM-Generated Code Quality with GRPOcited by 5
  25. 2026Outcome-Based RL Provably Leads Transformers to Reason, but Only With the Right Datacited by 5
  26. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Modelscited by 5
  27. 2025EyeMulator: Improving Code Language Models by Mimicking Human Visual Attentioncited by 4
  28. 2025Alignment with Fill-In-the-Middle for Enhancing Code Generationcited by 4
  29. 2025Reducing Belief Deviation in Reinforcement Learning for Active Reasoningcited by 4
  30. 2025QiMeng-SALV: Signal-Aware Learning for Verilog Code Generationcited by 4
  31. 2025DRAFT-RL: Multi-Agent Chain-of-Draft Reasoning for Reinforcement Learning-Enhanced LLMscited by 4
  32. 2026TerraFormer: Automated Infrastructure-as-Code with LLMs Fine-Tuned via Policy-Guided Verifier Feedbackcited by 4
  33. 2025GiFT: Gibbs Fine-Tuning for Code Generationcited by 3
  34. 2026Supervised Fine-Tuning versus Reinforcement Learning: A Study of Post-Training Methods for Large Language Modelscited by 3
  35. 2024Distilling Desired Comments for Enhanced Code Review with Large Language Modelscited by 2
  36. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMscited by 2
  37. 2025CRPE: Expanding The Reasoning Capability of Large Language Model for Code Generationcited by 2
  38. 2024Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Modelscited by 1

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Execution-based Code Generation using Deep Reinforcement Learning
  2. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
  3. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search
  4. 2025Teaching Language Models to Critique via Reinforcement Learning
  5. 2025Rethinking Verification for LLM Code Generation: From Generation to Testing
  6. 2026SpecCoder: Specification-Aware Code Generation with Curriculum Dual-Task Reinforcement Learning ↗
  7. 2026Performance, Efficiency and Collapse -- Advantages and Challenges in Offline Post-training of Code LLMs
  8. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback