# Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation

arXiv: https://arxiv.org/abs/2502.14948 · HTML: https://mlsanity.com/paper/2502.14948/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 20 Feb 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Code and instruction data synthesis](https://mlsanity.com/synth/code-and-instruction-data-synthesis/)

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Measuring Coding Challenge Competence With APPS](https://mlsanity.com/paper/2105.09938/) (2021)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [CodeT: Code Generation with Generated Tests](https://mlsanity.com/paper/2207.10397/) (2022)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [Magicoder: Empowering Code Generation with OSS-Instruct](https://mlsanity.com/paper/2312.02120/) (2023)
- [LLM Critics Help Catch LLM Bugs](https://mlsanity.com/paper/2407.00215/) (2024)
- [RLTF: Reinforcement Learning from Unit Test Feedback](https://mlsanity.com/paper/2307.04349/) (2023)
- [Gecko: Versatile Text Embeddings Distilled from Large Language Models](https://mlsanity.com/paper/2403.20327/) (2024)
- [Automated Unit Test Improvement using Large Language Models at Meta](https://mlsanity.com/paper/2402.09171/) (2024)
- [Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models](https://mlsanity.com/paper/2406.13542/) (2024)
- [CodeDPO: Aligning Code Models with Self Generated and Verified Source Code](https://mlsanity.com/paper/2410.05605/) (2024)
- [B4: Towards Optimal Assessment of Plausible Code Solutions with Plausible Tests](https://mlsanity.com/paper/2409.08692/) (2024)

## Built on by (most cited first, 30 of 35)

- [R-Zero: Self-Evolving Reasoning LLM from Zero Data](https://mlsanity.com/paper/2508.05004/) (2025)
- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoning](https://mlsanity.com/paper/2511.16043/) (2025)
- [Multi-Agent Evolve: LLM Self-Improve through Co-evolution](https://mlsanity.com/paper/2510.23595/) (2025)
- [Toward Training Superintelligent Software Agents through Self-Play SWE-RL](https://mlsanity.com/paper/2512.18552/) (2025)
- [Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2505.13445/) (2025)
- [Guided Self-Evolving LLMs with Minimal Human Supervision](https://mlsanity.com/paper/2512.02472/) (2025)
- [TTCS: Test-Time Curriculum Synthesis for Self-Evolving](https://mlsanity.com/paper/2601.22628/) (2026)
- [G-Zero: Self-Play for Open-Ended Generation from Zero Data](https://mlsanity.com/paper/2605.09959/) (2026)
- [Learning to Generate Unit Test via Adversarial Reinforcement Learning](https://mlsanity.com/paper/2508.21107/) (2025)
- [ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaboration](https://mlsanity.com/paper/2509.21823/) (2025)
- [R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Training](https://mlsanity.com/paper/2602.13103/) (2026)
- [SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms](https://mlsanity.com/paper/2506.06499/) (2025)
- [SEAD: Self-Evolving Agent for Multi-Turn Service Dialogue](https://mlsanity.com/paper/2602.03548/) (2026)
- [DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution](https://mlsanity.com/paper/2601.13761/) (2026)
- [ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization](https://mlsanity.com/paper/2605.16299/) (2026)
- [A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curricula](https://mlsanity.com/paper/2603.24202/) (2026)
- [Discretizing Reward Models](https://mlsanity.com/paper/2606.21795/) (2026)
- [HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learning](https://mlsanity.com/paper/2511.01104/) (2025)
- [TTSR: Test-Time Self-Evolving via Reflection](https://mlsanity.com/paper/2603.03297/) (2026)
- [Self-Evolving Coding Agents](https://mlsanity.com/paper/2608.03392/) (2026)
- [OptiHive: Ensemble Selection for LLM-Based Optimization via Statistical Modeling](https://mlsanity.com/paper/2508.02503/) (2025)
- [COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game](https://mlsanity.com/paper/2603.28386/) (2026)
- [Anchored Self-Play for Code Repair](https://mlsanity.com/paper/2607.03523/) (2026)
- [TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation](https://mlsanity.com/paper/2608.00494/) (2026)
- [Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs](https://mlsanity.com/paper/2609.03955/) (2026)
- [Sci-CoE: Co-evolving Scientific Reasoning LLMs via Geometric Consensus with Sparse Supervision](https://mlsanity.com/paper/2602.12164/) (2026)
- [Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Models](https://mlsanity.com/paper/2602.22227/) (2026)
- [Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verification](https://mlsanity.com/paper/2604.17010/) (2026)
- [ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoning](https://mlsanity.com/paper/2604.27644/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have improved their performance on coding benchmarks. However, improvement is plateauing due to the exhaustion of readily available high-quality data. Prior work has shown the potential of synthetic self-instruct data, but naively training on a model's own outputs can cause error accumulation, especially in coding tasks, where generalization may collapse due to overly simple or erroneous training data, highlighting the need for rigorous quality checks on synthetic data. In this work, we explore an effective approach whereby the model itself verifies the correctness of its own data. We thus propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity. By iteratively refining code (LLM-as-a-solver) and tests (LLM-as-a-verifier) together, we boost both capabilities without relying on human annotations or larger teacher models. Experiments with the Llama 3.1 8B model demonstrate substantial performance enhancements, achieving average relative improvements of 19.63% in code generation and 17.49% in test generation on MBPP and LiveCodeBench.