mlsanityData to 14 Oct 2026

CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment

arXiv 2510.18471 ↗First posted 21 Oct 2025cs.SE cs.AI cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

While Large Language Models (LLMs) excel at code generation by learning from vast code corpora, a fundamental semantic gap remains between their training on textual patterns and the goal of functional correctness, which is governed by formal execution semantics. Reinforcement Learning with Verifiable Rewards (RLVR) approaches attempt to bridge this gap using outcome rewards from executing test cases. However, solely relying on binary pass/fail signals is inefficient for establishing a well-aligned connection between the textual representation of code and its execution semantics, especially for subtle logical errors within the code. In this paper, we propose CodeRL+, a novel approach that integrates execution semantics alignment into the RLVR training pipeline for code generation. CodeRL+ enables the model to infer variable-level execution trajectory, providing a direct learning signal of execution semantics. CodeRL+ can construct execution semantics alignment directly using existing on-policy rollouts and integrates seamlessly with various RL algorithms. Extensive experiments demonstrate that CodeRL+ outperforms post-training baselines (including RLVR and Distillation), achieving a 4.6% average relative improvement in pass@1. CodeRL+ generalizes effectively to other coding tasks, yielding 15.5% and 4.4% higher accuracy on code-reasoning and test-output-generation benchmarks, respectively. CodeRL+ shows strong applicability across diverse RL algorithms and LLMs. Furthermore, probe analyses provide compelling evidence that CodeRL+ strengthens the alignment between code's textual representations and its underlying execution semantics.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 21 Oct 2025.

Builds on

40 of 54 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2021Program Synthesis with Large Language Modelscited by 3,637
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  9. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  11. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  12. 2024Qwen2.5-Coder Technical Reportcited by 1,203
28 more
  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2021Measuring Coding Challenge Competence With APPScited by 902
  3. 2019Designing and Interpreting Probes with Control Taskscited by 640
  4. 2025Process Reinforcement through Implicit Rewardscited by 504
  5. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 358
  6. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  7. 2023The False Promise of Imitating Proprietary LLMscited by 263
  8. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 262
  9. 2023Self-collaboration Code Generation via ChatGPTcited by 258
  10. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  11. 2023TACO: Topics in Algorithmic COde generation datasetcited by 160
  12. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 160
  13. 2020Predicting What You Already Know Helps: Provable Self-Supervised Learningcited by 152
  14. 2023Self-planning Code Generation with Large Language Modelscited by 124
  15. 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 122
  16. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  17. 2025A Survey on Code Generation with LLM-based Agentscited by 106
  18. 2025LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMscited by 70
  19. 2023CodeScore: Evaluating Code Generation by Learning Code Executioncited by 67
  20. 2024How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Studycited by 64
  21. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 56
  22. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 55
  23. 2025RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimizationcited by 35
  24. 2024Beyond Functional Correctness: Investigating Coding Style Inconsistencies in Large Language Modelscited by 15
  25. 2024ROCODE: Integrating Backtracking Mechanism and Program Analysis in Large Language Models for Code Generationcited by 13
  26. 2025ReCode: Updating Code API Knowledge with Reinforcement Learningcited by 12
  27. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RLcited by 7
  28. 2024VisualCoder: Guiding Large Language Models in Code Execution with Fine-grained Multimodal Chain-of-Thought Reasoningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 24.

24 papers.

Sort:

By area:

  1. 2026Code as Agent Harnesscited by 47
  2. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 31
  3. 2026Controlled Self-Evolution for Algorithmic Code Optimizationcited by 12
  4. 2026Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Taskscited by 10
  5. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skillscited by 10
  6. 2026CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Modelscited by 5
  7. 2026Execution-Grounded Credit Assignment for GRPO in Code Generationcited by 5
  8. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  9. 2026CoRe-Code: Collaborative Reinforcement Learning for Code Generationcited by 4
  10. 2026LAD: Learning Advantage Distribution for Reasoningcited by 2
  11. 2026SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Modelscited by 2
  12. 2026Harnessing Routing Foresight for Micro-step-level MoE load balancing in RL Post-trainingcited by 2
12 more
  1. 2026Turning Off-Policy Tokens On-Policy: A Plug-in Approach for Improving LLM Alignmentcited by 2
  2. 2026Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedbackcited by 1
  3. 2026DiDPO: Diff-in-Diff Policy Optimization for Coding Agent Trainingcited by 1
  4. 2026Context Bootstrapped Reinforcement Learningcited by 0
  5. 2026Schedule-and-Calibrate: Utility-Guided Multi-Task Reinforcement Learning for Code LLMscited by 0
  6. 2026Boosting Reinforcement Learning with Verifiable Rewards via Randomly Selected Few-Shot Guidancecited by 0
  7. 2026ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learningcited by 0
  8. 2026RLPF: Reinforcement Learning from Performance Feedback for Code Generationcited by 0
  9. 2026AI Control Scientist: LLM-driven Agentic System for Automated Control Designcited by 0
  10. 2026Improving Large Language Models for Code through Runtime Program-State Reasoningcited by 0
  11. 2026ReMCTS: Reflection-Enhanced Monte Carlo Tree Search for Code Generationcited by 0
  12. 2026VIGOR: Verifiable Industrial Geometry Optimization and Repair in Scene Programscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback
  2. 2026CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learning
  3. 2024Process Supervision-Guided Policy Optimization for Code Generation
  4. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Survey
  5. 2026ExecVerify: White-Box RL with Verifiable Stepwise Rewards for Code Execution Reasoning
  6. 2025Integrating Symbolic Execution into the Fine-Tuning of Code-Generating LLMs
  7. 2026Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generation
  8. 2026Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards