mlsanityData to 14 Oct 2026

Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization

arXiv 2505.23387 ↗First posted 29 May 2025cs.SE cs.AIcited by 18

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

also in 2 more
Abstract (arXiv)

Large Language Models (LLMs) generate functionally correct solutions but often fall short in code efficiency, a critical bottleneck for real-world deployment. In this paper, we introduce a novel test-time iterative optimization framework to address this, employing a closed-loop system where LLMs iteratively refine code based on empirical performance feedback from an execution sandbox. We explore three training strategies: Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO). Experiments on our Venus dataset and the APPS benchmark show that SFT and DPO rapidly saturate in efficiency gains. In contrast, GRPO, using reinforcement learning (RL) with execution feedback, continuously optimizes code performance, significantly boosting both pass@1 (from 47% to 62%) and the likelihood of outperforming human submissions in efficiency (from 31% to 45%). Our work demonstrates effective test-time code efficiency improvement and critically reveals the power of RL in teaching LLMs to truly self-improve code efficiency.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 52 of its 74 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

52 of 74 references.

References · 52

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2021Program Synthesis with Large Language Modelscited by 3,637
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  11. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  12. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
38 more
  1. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  2. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  4. 2021Measuring Coding Challenge Competence With APPScited by 902
  5. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  6. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  7. 2024A Survey on Large Language Models for Code Generationcited by 658
  8. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 546
  9. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 502
  10. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  11. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 300
  12. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  13. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 189
  14. 2023SantaCoder: don't reach for the stars!cited by 170
  15. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 151
  16. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 130
  17. 2024Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-stepcited by 127
  18. 2023Learning Performance-Improving Code Editscited by 106
  19. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 83
  20. 2023Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issuescited by 79
  21. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedbackcited by 77
  22. 2024Evaluating Language Models for Efficient Code Generationcited by 73
  23. 2024EffiBench: Benchmarking the Efficiency of Automatically Generated Codecited by 64
  24. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  25. 2024How Efficient is LLM-Generated Code? A Rigorous & High-Standard Benchmarkcited by 43
  26. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  27. 2024A Systematic Literature Review on Large Language Models for Automated Program Repaircited by 37
  28. 2024Mercury: A Code Efficiency Benchmark for Code Large Language Modelscited by 35
  29. 2024EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimizationcited by 30
  30. 2024ECCO: Can We Improve Model-Generated Code Efficiency Without Sacrificing Functional Correctness?cited by 27
  31. 2025EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Codecited by 27
  32. 2025Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contaminationcited by 23
  33. 2024CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programmingcited by 19
  34. 2024Efficient and Green Large Language Models for Software Engineering: Literature Review, Vision, and the Road Aheadcited by 17
  35. 2025Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Pointscited by 14
  36. 2024Measuring the Influence of Incorrect Code on Test Generationcited by 11
  37. 2025LLM4EFFI: Leveraging Large Language Models to Enhance Code Efficiency and Correctnesscited by 10
  38. 2025DynaCode: A Dynamic Complexity-Aware Code Benchmark for Evaluating Large Language Models in Code Generationcited by 7

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 14.

18 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic Systemcited by 15
  3. 2026Controlled Self-Evolution for Algorithmic Code Optimizationcited by 12
  4. 2026Secure Code Generation via Online Reinforcement Learning with Vulnerability Reward Modelcited by 6
  5. 2025Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environmentcited by 5
  6. 2026MaxCode: A Max-Reward Reinforcement Learning Framework for Automated Code Optimizationcited by 4
  7. 2026Extrapolative Weight Averaging Reveals Correctness-Efficiency Frontiers in Code RLcited by 2
  8. 2025Automated Design Optimization via Strategic Search with Large Language Modelscited by 1
  9. 2026Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoringcited by 1
  10. 2026ParEVO: Synthesizing Code for Irregular Data: High-Performance Parallelism through Agentic Evolutioncited by 0
  11. 2026An Initial Exploration of Contrastive Prompt Tuning to Generate Energy-Efficient Codecited by 0
  12. 2026Paper Espresso: From Paper Overload to Research Insightcited by 0
6 more
  1. 2026AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Codecited by 0
  2. 2026Mastermind: Strategy-grounded Learning for Repository-Scale Vulnerability Reproductioncited by 0
  3. 2026Beyond the Need for Speed: Energy-Aware Code Generation via Simulation-Guided Reinforcement Learningcited by 0
  4. 2026Reinforcement Learning for Code Optimizationcited by 0
  5. 2026Collective Bias Mitigation via Model Routing and Collaborationcited by 0
  6. 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue Resolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Better Correctness and Efficiency in Code Generation
  2. 2025From Reasoning to Code: GRPO Optimization for Underrepresented Languages
  3. 2026Breaking Training Bottlenecks: Effective and Stable Reinforcement Learning for Coding Models
  4. 2026RLPF: Reinforcement Learning from Performance Feedback for Code Generation
  5. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
  6. 2025On-Policy Optimization with Group Equivalent Preference for Multi-Programming Language Understanding
  7. 2024Process Supervision-Guided Policy Optimization for Code Generation
  8. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs