mlsanityData to 7 Oct 2026

Execution-based Code Generation using Deep Reinforcement Learning

arXiv 2301.13816 ↗First posted 31 Jan 2023cs.LG cs.AI cs.CLcited by 95

Not on the map. Why?

Abstract (arXiv)

The utilization of programming language (PL) models, pre-trained on large-scale code corpora, as a means of automating software engineering processes has demonstrated considerable potential in streamlining various code generation tasks such as code completion, code translation, and program synthesis. However, current approaches mainly rely on supervised fine-tuning objectives borrowed from text generation, neglecting unique sequence-level characteristics of code, including but not limited to compilability as well as syntactic and functional correctness. To address this limitation, we propose PPOCoder, a new framework for code generation that synergistically combines pre-trained PL models with Proximal Policy Optimization (PPO) which is a widely used deep reinforcement learning technique. By utilizing non-differentiable feedback from code execution and structure alignment, PPOCoder seamlessly integrates external code-specific knowledge into the model optimization process. It's important to note that PPOCoder is a task-agnostic and model-agnostic framework that can be used across different code generation tasks and PLs. Extensive experiments on three code generation tasks demonstrate the effectiveness of our proposed approach compared to SOTA methods, achieving significant improvements in compilation success rates and functional correctness across different PLs.

Key facts. Cited by 95 papers in the map (data through 7 Oct 2026). Builds on 42 of its 52 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

42 of 52 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2021Program Synthesis with Large Language Modelscited by 3,509
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  12. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,024
30 more
  1. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  2. 2021Measuring Coding Challenge Competence With APPScited by 879
  3. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  4. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  5. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  6. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 638
  7. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  8. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  9. 2020CodeBLEU: a Method for Automatic Evaluation of Code Synthesiscited by 468
  10. 2021Unified Pre-training for Program Understanding and Generationcited by 361
  11. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  12. 2020Unsupervised Translation of Programming Languagescited by 262
  13. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  14. 2019SPoC: Search-based Pseudocode to Codecited by 176
  15. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  16. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 144
  17. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  18. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  19. 2019Write, Execute, Assess: Program Synthesis with a REPLcited by 92
  20. 2020Code Prediction by Feeding Trees to Transformerscited by 70
  21. 2022XLCoST: A Benchmark Dataset for Cross-lingual Code Intelligencecited by 54
  22. 2017Code Completion with Neural Attention and Pointer Networkscited by 49
  23. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 48
  24. 2022Compilable Neural Code Generation with Compiler Feedbackcited by 46
  25. 2019Learning to Infer Program Sketchescited by 44
  26. 2021Latent Execution for Neural Program Synthesiscited by 35
  27. 2022StructCoder: Structure-Aware Transformer for Code Generationcited by 33
  28. 2021Code Completion by Modeling Flattened Abstract Syntax Trees as Graphscited by 30
  29. 2022Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Codecited by 20
  30. 2021Energy-Based Models for Code Generation under Compilability Constraintscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 21; 2025: 28; 2026: 29.

95 papers.

Sort:

By area:

  1. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  2. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 117
  3. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 110
  4. 2024Large Language Models Meet NLP: A Surveycited by 102
  5. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 83
  6. 2023Transformer-based Planning for Symbolic Regressioncited by 72
  7. 2023PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedbackcited by 69
  8. 2024Agents in Software Engineering: Survey, Landscape, and Visioncited by 60
  9. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  10. 2025FLEX: Continuous Agent Evolution via Forward Learning from Experiencecited by 51
  11. 2024LeDex: Training LLMs to Better Self-Debug and Explain Codecited by 39
  12. 2025CodeContests+: High-Quality Test Case Generation for Competitive Programmingcited by 39
38 more
  1. 2025Navigate the Unknown: Enhancing LLM Reasoning with Intrinsic Motivation Guided Explorationcited by 33
  2. 2024AST-T5: Structure-Aware Pretraining for Code Generation and Understandingcited by 32
  3. 2024Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learningcited by 32
  4. 2023LLM-Assisted Code Cleaning For Training Accurate Code Generatorscited by 27
  5. 2025Multi-Turn Code Generation Through Single-Step Rewardscited by 27
  6. 2024Process Supervision-Guided Policy Optimization for Code Generationcited by 25
  7. 2023CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Executioncited by 21
  8. 2023Tool-Augmented Reward Modelingcited by 20
  9. 2025Improving RL Exploration for LLM Reasoning through Retrospective Replaycited by 19
  10. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  11. 2023B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesiscited by 16
  12. 2024Search, Verify and Feedback: Towards Next Generation Post-training Paradigm of Foundation Models via Verifier Engineeringcited by 16
  13. 2024MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actionscited by 15
  14. 2023Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decodingcited by 14
  15. 2025Insights from Verification: Training a Verilog Generation LLM with Reinforcement Learning with Testbench Feedbackcited by 14
  16. 2023Explaining Competitive-Level Programming Solutions using LLMscited by 13
  17. 2024RLSF: Fine-tuning LLMs via Symbolic Feedbackcited by 13
  18. 2023Fine-tune Language Models to Approximate Unbiased In-context Learningcited by 12
  19. 2023Fine-Tuning Language Models Using Formal Methods Feedbackcited by 12
  20. 2025Rethinking Verification for LLM Code Generation: From Generation to Testingcited by 12
  21. 2023Reinforcement Learning for Generative AI: A Surveycited by 11
  22. 2023Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challengescited by 10
  23. 2024DSTC: Direct Preference Learning with Only Self-Generated Tests and Code to Improve Code LMscited by 9
  24. 2025Klear-CodeTest: Scalable Test Case Generation for Code Reinforcement Learningcited by 9
  25. 2023PerfRL: A Small Language Model Framework for Efficient Code Optimizationcited by 8
  26. 2024PLUM: Improving Code LMs with Execution-Guided On-Policy Preference Learning Driven By Synthetic Test Casescited by 8
  27. 2024FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization systemcited by 8
  28. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RLcited by 7
  29. 2025Critique-Coder: Enhancing Coder Models by Critique Reinforcement Learningcited by 7
  30. 2026VisRefiner: Learning from Visual Differences for Screenshot-to-Code Generationcited by 7
  31. 2024CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinementcited by 6
  32. 2026Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learningcited by 6
  33. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  34. 2025The KoLMogorov Test: Compression by Code Generationcited by 5
  35. 2025LLM Benchmarking with LLaMA2: Evaluating Code Development Performance Across Multiple Programming Languagescited by 5
  36. 2025SuperCoder: Assembly Program Superoptimization with Large Language Modelscited by 5
  37. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Modelscited by 5
  38. 2026Execution-Grounded Credit Assignment for GRPO in Code Generationcited by 5

50 most cited of 95 All 95 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
  2. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment
  3. 2025Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization
  4. 2025CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
  5. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
  6. 2026StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning
  7. 2024Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement
  8. 2024Performance-Aligned LLMs for Generating Fast Code