# RLPF: Reinforcement Learning from Performance Feedback for Code Generation

arXiv: https://arxiv.org/abs/2607.27271 · HTML: https://mlsanity.com/paper/2607.27271/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 29 Jul 2026.

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [RLTF: Reinforcement Learning from Unit Test Feedback](https://mlsanity.com/paper/2307.04349/) (2023)
- [Evaluating Language Models for Efficient Code Generation](https://mlsanity.com/paper/2408.06450/) (2024)
- [PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback](https://mlsanity.com/paper/2307.14936/) (2023)
- [EffiBench: Benchmarking the Efficiency of Automatically Generated Code](https://mlsanity.com/paper/2402.02037/) (2024)
- [CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning](https://mlsanity.com/paper/2507.14111/) (2025)
- [EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code](https://mlsanity.com/paper/2505.13004/) (2025)
- [PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedback](https://mlsanity.com/paper/2412.03578/) (2024)
- [CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment](https://mlsanity.com/paper/2510.18471/) (2025)
- [COFFE: A Code Efficiency Benchmark for Code Generation](https://mlsanity.com/paper/2502.02827/) (2025)
- [B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis](https://mlsanity.com/paper/2310.03173/) (2023)
- [EffiCoder: Enhancing Code Generation in Large Language Models through Efficiency-Aware Fine-tuning](https://mlsanity.com/paper/2410.10209/) (2024)
- [DevBench: A Realistic, Developer-Informed Benchmark for Code Generation Models](https://mlsanity.com/paper/2601.11895/) (2026)
- [CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models](https://mlsanity.com/paper/2602.17684/) (2026)
- [MaxCode: A Max-Reward Reinforcement Learning Framework for Automated Code Optimization](https://mlsanity.com/paper/2601.05475/) (2026)
- [CoRe-Code: Collaborative Reinforcement Learning for Code Generation](https://mlsanity.com/paper/2605.24812/) (2026)
- [Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Tree](https://mlsanity.com/paper/2601.14523/) (2026)
- [PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization](https://mlsanity.com/paper/2605.15222/) (2026)
- [Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback](https://mlsanity.com/paper/2605.30478/) (2026)

## Abstract (arXiv)

Code models are increasingly trained with execution feedback, but most training signals still stop at correctness. This leaves an important gap for systems code: two programs can pass the same tests while differing greatly in runtime. We study how to train code agents to prefer faster correct implementations, rather than treating efficiency only as an evaluation metric. The key difficulty is that runtime is a fragile reward. It is meaningful only after a program is correct, varies across tasks, and gives little guidance when most sampled programs fail to compile or run. We propose RLPF, reinforcement learning from performance feedback, which turns execution outcomes into a staged reward. Failed programs are ordered by execution progress, while correct programs are ranked by their relative improvement from the baseline toward the expert reference. This gives useful feedback before correctness and performance-sensitive feedback after correctness. Fine-tuning Qwen3-32B with RLPF on PerfCodeBench raises correct-and-runnable solutions from 11.1% to 54.6% and improves relative efficiency from 8.1% to 38.6%. The trained model becomes competitive with stronger open-weight systems, and its optimization behavior transfers modestly to EffiBench-X. Additional studies show that model-generated references provide useful but weaker supervision, and that the full composite reward is more reliable than correctness-only or runtime-only baselines. These results suggest that code agents can be trained not only to pass tests, but also to optimize the programs they write.