Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce

arXiv 2608.28638 ↗First posted 10 Aug 2026cs.AIcited by 0

On the map: Synthetic data and self-improvement for LLMs → Self-evolving LLM agents

Abstract (arXiv)

Agent skills are portable packages of instructions and resources an agent consults at deployment. Self-evolving them fails in two ways today. First, skills evolved from scratch underperform human-curated ones and, on a weak model, using no skill at all. Second, an evolution-time pass records one lucky trajectory that a fresh stochastic agent often fails to reproduce at deployment. We present reSolve, a per-task, oracle-in-the-loop framework built on three components. It decouples interactive solving from a self-contained deliverable that is independently re-executed in a fresh container, a protocol we call solve-and-reproduce. It enhances the sparse reward signal with a surrogate verifier that cannot access hidden tests or reference answers. It then runs verifier-guided beam search over a solution-construction graph. Within a fixed harness, a cheap model self-evolves skills that reach 74.9% mean-of-3, +14.8 points over the 60.1% human-curated baseline, exceeding the strongest official curated-skill result (67.3%, GPT-5.5/OpenHands). We also report observed failure cases and domain-level results, including performance on the 14 Natural Science tasks, to clarify when the approach does and does not help.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Self-evolving LLM agents, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 20 of its 26 references that are in the map. First posted on arXiv 10 Aug 2026.

Builds on

20 of 26 references.

Closest ancestors · 1

Same direction, newest first

2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification2 Apr

Other references · 19

2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 574
2024Executable Code Actions Elicit Better LLM Agentscited by 564
2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 392
2024Agent Workflow Memorycited by 383

7 more

2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
2025SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skillscited by 192
2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 187
2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 130
2026SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditingcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way