mlsanityData to 7 Oct 2026

Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gain

arXiv 2603.02218 ↗First posted 10 Feb 2026cs.LG cs.AI cs.CLcited by 16

On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents

Abstract (arXiv)

Large language models (LLMs) make it plausible to build systems that improve through self-evolving loops, but many existing proposals are better understood as self-play and often plateau quickly. A central failure mode is that the loop synthesises more data without increasing learnable information for the next iteration. Through experiments on a self-play coding task, we reveal that sustainable self-evolution requires a self-synthesised data pipeline with learnable information that increases across iterations. We identify triadic roles that self-evolving LLMs play: the Proposer, which generates tasks; the Solver, which attempts solutions; and the Verifier, which provides training signals, and we identify three system designs that jointly target learnable information gain from this triadic roles perspective. Asymmetric co-evolution closes a weak-to-strong-to-weak loop across roles. Capacity growth expands parameter and inference-time budgets to match rising learnable information. Proactive information seeking introduces external context and new task sources that prevent saturation. Together, these modules provide a measurable, system-level path from brittle self-play dynamics to sustained self-evolution.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 42 of its 60 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

42 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolution27 Oct
  2. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems10 Aug

Other references · 40

  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  2. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  3. 2024Self-Rewarding Language Modelscited by 654
  4. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  5. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  6. 2023Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervisioncited by 445
  7. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  8. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  9. 2025Memory in the Age of AI Agentscited by 285
  10. 2025TTRL: Test-Time Reinforcement Learningcited by 253
  11. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  12. 2023Do Large Language Models Know What They Don't Know?cited by 238
28 more
  1. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 210
  2. 2025Learning to Reason without External Rewardscited by 199
  3. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  4. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improvecited by 162
  5. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  6. 2025Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computationcited by 120
  7. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 83
  8. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 81
  9. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 78
  10. 2025Reinforcing General Reasoning without Verifierscited by 76
  11. 2024MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMscited by 72
  12. 2024Harder Tasks Need More Experts: Dynamic Routing in MoE Modelscited by 71
  13. 2025Self-Questioning Language Modelscited by 47
  14. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoningcited by 43
  15. 2025EvolveSearch: An Iterative Self-Evolving Search Agentcited by 42
  16. 2024Self-Boosting Large Language Models with Synthetic Preference Datacited by 40
  17. 2025RL Tango: Reinforcing Generator and Verifier Together for Language Reasoningcited by 39
  18. 2026From Entropy to Epiplexity: Rethinking Information for Computationally Bounded Intelligencecited by 39
  19. 2025Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigationcited by 36
  20. 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulatorscited by 34
  21. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  22. 2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
  23. 2025PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifiercited by 28
  24. 2025Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolutioncited by 24
  25. 2025OpenSIR: Open-Ended Self-Improving Reasonercited by 7
  26. 2025From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMscited by 5
  27. 2025Curriculum-Guided Layer Scaling for Language Model Pretrainingcited by 4
  28. 2025Progressive Scaling Visual Object Trackingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

  1. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  2. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  3. 2026OpenRCA 2.0: From Outcome Labels to Causal Process Supervisioncited by 5
  4. 2026Survive or Collapse: The Asymmetric Roles of Data Gating and Reward Grounding in Self-Play RLcited by 4
  5. 2026SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Taskscited by 4
  6. 2026To See is Not to Master: Teaching LLMs to Use Private Libraries for Code Generationcited by 3
  7. 2026EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformationscited by 3
  8. 2026What Do Evolutionary Coding Agents Evolve?cited by 3
  9. 2026Towards Self-Improving Error Diagnosis in Multi-Agent Systemscited by 2
  10. 2026RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agentscited by 2
  11. 2026Self-Evolving Coding Agentscited by 2
  12. 2026Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolutioncited by 1
4 more
  1. 2026Confidence-Orchestrated Self-Evolution against Uncertain LLM Feedbackcited by 0
  2. 2026Closed-Loop Knowledge Dynamics: An Operational Framework for Saturation and Escapecited by 0
  3. 2026J-Zero: Unified Challenger--Solver--Judge Self-Evolution from Zero Datacited by 0
  4. 2026STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  2. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  3. 2026Skill Self-Play: Pushing the Frontier of LLM Capability with Co-Evolving Skills
  4. 2026Learning What to Remember and What to Internalize in LLM Self-Evolution via Adaptive Memory-Parameter Coordination
  5. 2026Self-Consolidation for Self-Evolving Agents
  6. 2026SEIF: Self-Evolving Reinforcement Learning for Instruction Following
  7. 2026When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
  8. 2026Large Language Model Agents Are Not Always Faithful Self-Evolvers