mlsanityData to 14 Oct 2026

Improving LLM-Generated Code Quality with GRPO

arXiv 2506.02211 ↗First posted 2 Jun 2025cs.AIcited by 8

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Language Models (LLMs) are gaining widespread use for code generation. Recent training procedures use execution feedback as a reward signal, typically focusing on the functional correctness of the code, using unit test pass rate as a reward signal. However, this reward signal fails to capture notions of maintainability, quality and safety of the code produced. We address this under-explored area and develop a comprehensive library to quantify various aspects of code quality, and use it as a reward in GRPO. We find GRPO increases code quality according to this measure, which is confirmed by expert, blinded human annotators.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 15 of its 34 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

15 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning2 Oct
  2. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback2 Feb

Other references · 13

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2021Measuring Coding Challenge Competence With APPScited by 902
  8. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  9. 20242 OLMo 2 Furiouscited by 340
  10. 2024Process Supervision-Guided Policy Optimization for Code Generationcited by 26
  11. 2025Process-Supervised Reinforcement Learning for Code Generationcited by 22
  12. 2024ACECode: A Reinforcement Learning Framework for Aligning Code Efficiency and Correctness in Code Language Modelscited by 10
1 more
  1. 2025Iterative Self-Training for Code Generation via Reinforced Re-Rankingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 6.

8 papers.

  1. 2025FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learningcited by 9
  2. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Modelscited by 5
  3. 2026CoRe-Code: Collaborative Reinforcement Learning for Code Generationcited by 4
  4. 2026Find, Fix, Reason: Context Repair for Video Reasoningcited by 3
  5. 2026Debate as Reward: A Multi-Agent Reward System for Scientific Ideation via RL Post-Trainingcited by 3
  6. 2025Information-Consistent Language Model Recommendations through Group Relative Policy Optimizationcited by 2
  7. 2026Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Codecited by 1
  8. 2026LintEval: Benchmarking LLMs on Linter-Guided File Refactoringcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Training Language Models to Generate Quality Code with Program Analysis Feedback
  2. 2025Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation
  3. 2025Rethinking Verification for LLM Code Generation: From Generation to Testing
  4. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
  5. 2025Holistic Evaluation of State-of-the-Art LLMs for Code Generation
  6. 2025Enhancing the Non-Functional Quality Compliance of LLM-Generated Code through Quality-Aware Preference Learning
  7. 2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories
  8. 2025Evaluating Large Language Models for Functional and Maintainable Code in Industrial Settings: A Case Study at ASML