Self-Improving AI Agents through Self-Play
Not on the map. Why?
Abstract (arXiv)
We extend the moduli-theoretic framework of psychometric batteries to the domain of dynamical systems. While previous work established the AAI capability score as a static functional on the space of agent representations, this paper formalizes the agent as a flow nur parameterized by computational resource r, governed by a recursive Generator-Verifier-Updater (GVU) operator. We prove that this operator generates a vector field on the parameter manifold Theta, and we identify the coefficient of self-improvement kappa as the Lie derivative of the capability functional along this flow. The central contribution of this work is the derivation of the Variance Inequality, a spectral condition that is sufficient (under mild regularity) for the stability of self-improvement. We show that a sufficient condition for kappa > 0 is that, up to curvature and step-size effects, the combined noise of generation and verification must be small enough. We then apply this formalism to unify the recent literature on Language Self-Play (LSP), Self-Correction, and Synthetic Data bootstrapping. We demonstrate that architectures such as STaR, SPIN, Reflexion, GANs and AlphaZero are specific topological realizations of the GVU operator that satisfy the Variance Inequality through filtration, adversarial discrimination, or grounding in formal systems.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 14 of its 16 references that are in the map. First posted on arXiv 2 Dec 2025.
Builds on
14 of 16 references.
References · 14
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Let's Verify Step by Stepcited by 3,789
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
- 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
- 2025Language Self-Play For Data-Free Trainingcited by 43
- 2025SPELL: Self-Play Reinforcement Learning for Evolving Long-Context Language Modelscited by 29
Built on by
Citing papers per year: 2025: 2.
2 papers.
- 2025The Geometry of Benchmarks: A New Path Toward AGIcited by 0
- 2025Mathematics and Coding are Universal AI Benchmarkscited by 0
- 2025Mathematics and Coding are Universal AI Benchmarks15 Dec 2025
- 2025The Geometry of Benchmarks: A New Path Toward AGI3 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2025On The Statistical Limits of Self-Improving Agents
- 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
- 2026Self-Improvements in Modern Agentic Systems: A Survey
- 2026Self-Reference in Large Language Models: The Introspection Threshold for Recursive Self-Improvement
- 2026AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-Improvement
- 2026The Red Queen G"odel Machine: Co-Evolving Agents and Their Evaluators
- 2026Self-Aware Recursively Self-Improving Agents for Personal Singularity: A Goal-, Scope-, Tool-, and Benchmark-Driven Multi-Agent Architecture
- 2025Coherence Mechanisms for Provable Self-Improvement