# Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning

arXiv: https://arxiv.org/abs/2506.03136 · HTML: https://mlsanity.com/paper/2506.03136/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 57 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 30 of 43)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation](https://mlsanity.com/paper/2305.01210/) (2023)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [SimPO: Simple Preference Optimization with a Reference-Free Reward](https://mlsanity.com/paper/2405.14734/) (2024)
- [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://mlsanity.com/paper/2503.24290/) (2025)
- [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://mlsanity.com/paper/2401.08417/) (2024)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2502.03373/) (2025)
- [MMaDA: Multimodal Large Diffusion Language Models](https://mlsanity.com/paper/2505.15809/) (2025)
- [Inference-Time Scaling for Generalist Reward Modeling](https://mlsanity.com/paper/2504.02495/) (2025)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [IPO: Interior-point Policy Optimization under Constraints](https://mlsanity.com/paper/1910.09615/) (2019)
- [MaxMin-RLHF: Alignment with Diverse Human Preferences](https://mlsanity.com/paper/2402.08925/) (2024)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering](https://mlsanity.com/paper/2401.08500/) (2024)
- [S*: Test Time Scaling for Code Generation](https://mlsanity.com/paper/2502.14382/) (2025)
- [GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning](https://mlsanity.com/paper/2504.00891/) (2025)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)

## Built on by (most cited first, 30 of 56)

- [R-Zero: Self-Evolving Reasoning LLM from Zero Data](https://mlsanity.com/paper/2508.05004/) (2025)
- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [OpenClaw-RL: Train Any Agent Simply by Talking](https://mlsanity.com/paper/2603.10165/) (2026)
- [Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning](https://mlsanity.com/paper/2511.16043/) (2025)
- [WoW: Towards a World omniscient World model Through Embodied Interaction](https://mlsanity.com/paper/2509.22642/) (2025)
- [ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2506.18896/) (2025)
- [GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators](https://mlsanity.com/paper/2512.19682/) (2025)
- [Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models](https://mlsanity.com/paper/2506.07468/) (2025)
- [Guided Self-Evolving LLMs with Minimal Human Supervision](https://mlsanity.com/paper/2512.02472/) (2025)
- [Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data](https://mlsanity.com/paper/2602.21320/) (2026)
- [Demystifying Reinforcement Learning in Agentic Reasoning](https://mlsanity.com/paper/2510.11701/) (2025)
- [Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs](https://mlsanity.com/paper/2510.11062/) (2025)
- [CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment](https://mlsanity.com/paper/2510.18471/) (2025)
- [RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL System](https://mlsanity.com/paper/2602.02488/) (2026)
- [A Survey on Agentic Multimodal Large Language Models](https://mlsanity.com/paper/2510.10991/) (2025)
- [SOD: Step-wise On-policy Distillation for Small Language Model Agents](https://mlsanity.com/paper/2605.07725/) (2026)
- [Benchmarking Reward Hack Detection in Code Environments via Contrastive Analysis](https://mlsanity.com/paper/2601.20103/) (2026)
- [The Best of N Worlds: Aligning Reinforcement Learning with Best-of-N Sampling via max@k Optimisation](https://mlsanity.com/paper/2510.23393/) (2025)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [Loong: Synthesize Long Chain-of-Thoughts at Scale through Verifiers](https://mlsanity.com/paper/2509.03059/) (2025)
- [V1: Unifying Generation and Self-Verification for Parallel Reasoners](https://mlsanity.com/paper/2603.04304/) (2026)
- [Learning to Generate Unit Test via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2508.21107/) (2025)
- [R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Training](https://mlsanity.com/paper/2602.13103/) (2026)
- [The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback](https://mlsanity.com/paper/2510.02752/) (2025)
- [VideoChat-M1: Collaborative Policy Planning for Video Understanding via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2511.19524/) (2025)
- [TestExplora: Benchmarking LLMs for Proactive Bug Discovery via Repository-Level Test Generation](https://mlsanity.com/paper/2602.10471/) (2026)
- [CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models](https://mlsanity.com/paper/2602.17684/) (2026)
- [Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence](https://mlsanity.com/paper/2608.31075/) (2026)
- [How Many Code and Test Cases Are Enough? Evaluating Test Cases Generation from a Binary-Matrix Perspective](https://mlsanity.com/paper/2510.08720/) (2025)

## Abstract (arXiv)

We propose CURE, a novel reinforcement learning framework with a dedicated reward design that co-evolves coding and unit test generation capabilities based on their interaction outcomes, without any ground-truth code as supervision. This approach enables flexible and scalable training and allows the unit tester to learn directly from the coder's mistakes. Our derived ReasonFlux-Coder-7B and 14B models improve code generation accuracy by 5.3% and Best-of-N accuracy by 9.0% after optimization on Qwen2.5-Instruct models, outperforming similarly sized Qwen-Coder, DeepSeek-Coder, and Seed-Coder. They naturally extend to downstream tasks such as test-time scaling and agentic coding-achieving a 8.1% improvement over the base model. For the long-CoT model, our ReasonFlux-Coder-4B consistently outperforms Qwen3-4B while achieving 64.8% inference efficiency in unit test generation. Notably, we also find that our model can serve as an effective reward model for reinforcement learning on base models. Project: https://github.com/Gen-Verse/CURE