mlsanityData to 14 Oct 2026

Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation

arXiv 2502.14948 ↗First posted 20 Feb 2025cs.SEcited by 35

On the map: Synthetic data and self-improvement for LLMs → Code and instruction data synthesis

Abstract (arXiv)

Recent advances in large language models (LLMs) have improved their performance on coding benchmarks. However, improvement is plateauing due to the exhaustion of readily available high-quality data. Prior work has shown the potential of synthetic self-instruct data, but naively training on a model's own outputs can cause error accumulation, especially in coding tasks, where generalization may collapse due to overly simple or erroneous training data, highlighting the need for rigorous quality checks on synthetic data. In this work, we explore an effective approach whereby the model itself verifies the correctness of its own data. We thus propose Sol-Ver, a self-play solver-verifier framework that jointly improves a single model's code and test generation capacity. By iteratively refining code (LLM-as-a-solver) and tests (LLM-as-a-verifier) together, we boost both capabilities without relying on human annotations or larger teacher models. Experiments with the Llama 3.1 8B model demonstrate substantial performance enhancements, achieving average relative improvements of 19.63% in code generation and 17.49% in test generation on MBPP and LiveCodeBench.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Code and instruction data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 25 of its 32 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

25 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models19 Jun
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2021Program Synthesis with Large Language Modelscited by 3,637
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  9. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  11. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  12. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
11 more
  1. 2021Measuring Coding Challenge Competence With APPScited by 902
  2. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
  3. 2022CodeT: Code Generation with Generated Testscited by 384
  4. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  5. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 256
  6. 2024LLM Critics Help Catch LLM Bugscited by 147
  7. 2023RLTF: Reinforcement Learning from Unit Test Feedbackcited by 114
  8. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 83
  9. 2024Automated Unit Test Improvement using Large Language Models at Metacited by 75
  10. 2024CodeDPO: Aligning Code Models with Self Generated and Verified Source Codecited by 34
  11. 2024B4: Towards Optimal Assessment of Plausible Code Solutions with Plausible Testscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 23.

35 papers.

Sort:

By area:

  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 170
  3. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  4. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 60
  5. 2025Toward Training Superintelligent Software Agents through Self-Play SWE-RLcited by 40
  6. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  7. 2025Guided Self-Evolving LLMs with Minimal Human Supervisioncited by 35
  8. 2026TTCS: Test-Time Curriculum Synthesis for Self-Evolvingcited by 19
  9. 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
  10. 2025Learning to Generate Unit Test via Adversarial Reinforcement Learningcited by 11
  11. 2025ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaborationcited by 11
  12. 2026R-Diverse: Mitigating Diversity Illusion in Self-Play LLM Trainingcited by 10
23 more
  1. 2025SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithmscited by 7
  2. 2026SEAD: Self-Evolving Agent for Multi-Turn Service Dialoguecited by 7
  3. 2026DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolutioncited by 6
  4. 2026ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimizationcited by 4
  5. 2026A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curriculacited by 3
  6. 2026Discretizing Reward Modelscited by 3
  7. 2025HarnessLLM: Automatic Testing Harness Generation via Reinforcement Learningcited by 2
  8. 2026TTSR: Test-Time Self-Evolving via Reflectioncited by 2
  9. 2026Self-Evolving Coding Agentscited by 2
  10. 2025OptiHive: Ensemble Selection for LLM-Based Optimization via Statistical Modelingcited by 1
  11. 2026COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Gamecited by 1
  12. 2026Anchored Self-Play for Code Repaircited by 1
  13. 2026TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generationcited by 1
  14. 2026Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMscited by 1
  15. 2026Sci-CoE: Co-evolving Scientific Reasoning LLMs via Geometric Consensus with Sparse Supervisioncited by 0
  16. 2026Dynamic Adversarial Reinforcement Learning for Robust Multimodal Large Language Modelscited by 0
  17. 2026Improving LLM Code Reasoning via Semantic Equivalence Self-Play with Formal Verificationcited by 0
  18. 2026ANCORA: Learning to Question via Manifold-Anchored Self-Play for Verifiable Reasoningcited by 0
  19. 2026A2RBench: An Automatic Paradigm for Formally Verifiable Abstract Reasoning Benchmark Generationcited by 0
  20. 2026FABSVer: Faster Training and Better Self-Verification for LLM Mathematical Reasoningcited by 0
  21. 2026Robust Code RL via Faulty-Code-Driven Test case Synthesis and Dense Reward Shapingcited by 0
  22. 2026Learning to Optimize through Solver-Grounded Self-Playcited by 0
  23. 2026Self-Evolving Visual Agent via Margin Rewardcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Embarrassingly Simple Self-Distillation Improves Code Generation
  2. 2025Propose, Solve, Verify: Self-Play Through Formal Verification
  3. 2025HardTests: Synthesizing High-Quality Test Cases for LLM Coding
  4. 2025Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
  5. 2025Rethinking Verification for LLM Code Generation: From Generation to Testing
  6. 2026CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
  7. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code
  8. 2023SelfEvolve: A Code Evolution Framework via Large Language Models