mlsanityData to 14 Oct 2026

AgentEvolver: Towards Efficient Self-Evolving Agent System

arXiv 2511.10395 ↗First posted 13 Nov 2025cs.LG cs.AI cs.CLcited by 85

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Autonomous agents powered by large language models (LLMs) have the potential to significantly enhance human productivity by reasoning, using tools, and executing complex tasks in diverse environments. However, current approaches to developing such agents remain costly and inefficient, as they typically require manually constructed task datasets and reinforcement learning (RL) pipelines with extensive random exploration. These limitations lead to prohibitively high data-construction costs, low exploration efficiency, and poor sample utilization. To address these challenges, we present AgentEvolver, a self-evolving agent system that leverages the semantic understanding and reasoning capabilities of LLMs to drive autonomous agent learning. AgentEvolver introduces three synergistic mechanisms: (i) self-questioning, which enables curiosity-driven task generation in novel environments, reducing dependence on handcrafted datasets; (ii) self-navigating, which improves exploration efficiency through experience reuse and hybrid policy guidance; and (iii) self-attributing, which enhances sample efficiency by assigning differentiated rewards to trajectory states and actions based on their contribution. By integrating these mechanisms into a unified framework, AgentEvolver enables scalable, cost-effective, and continual improvement of agent capabilities. Preliminary experiments indicate that AgentEvolver achieves more efficient exploration, better sample utilization, and faster adaptation compared to traditional RL-based baselines.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 20 of its 34 references that are in the map. First posted on arXiv 13 Nov 2025.

Builds on

20 of 34 references.

References · 20

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2024Qwen2 Technical Reportcited by 2,120
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  5. 2018Exploration by Random Network Distillationcited by 993
  6. 2024DeepSeek-V3 Technical Reportcited by 811
  7. 2025Process Reinforcement through Implicit Rewardscited by 504
  8. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  9. 2025Learning to Reason under Off-Policy Guidancecited by 281
  10. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  11. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 200
  12. 2025ToRL: Scaling Tool-Integrated RLcited by 169
8 more
  1. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 158
  2. 2025Outcome-based Exploration for LLM Reasoningcited by 83
  3. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Modelscited by 59
  4. 2024Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Modelscited by 33
  5. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfallscited by 26
  6. 2025Tool-R1: Sample-Efficient Reinforcement Learning for Agentic Tool Usecited by 16
  7. 2025THOR: Tool-Integrated Hierarchical Optimization via RL for Mathematical Reasoningcited by 9
  8. 2025Modeling Unseen Environments with Language-guided Composable Causal Components in Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 82.

85 papers.

Sort:

By area:

  1. 2025Memory in the Age of AI Agentscited by 303
  2. 2026MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agentscited by 142
  3. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 90
  4. 2026SkillClaw: Let Skills Evolve Collectively with Agentic Evolvercited by 81
  5. 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
  6. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
  7. 2026Dynamic Dual-Granularity Skill Bank for Agentic RLcited by 34
  8. 2026Graph-based Agent Memory: Taxonomy, Techniques, and Applicationscited by 25
  9. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 21
  10. 2026A Comprehensive Survey on Agent Skills: Taxonomy, Techniques, and Applicationscited by 18
  11. 2026Position: Agentic Evolution is the Path to Evolving LLMscited by 16
  12. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 15
38 more
  1. 2026GEAR: Granularity-Adaptive Advantage Reweighting for LLM Agents via Self-Distillationcited by 12
  2. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 12
  3. 2026From Question Answering to Task Completion: A Survey on Agent System and Harness Designcited by 12
  4. 2026GenericAgent: A Token-Efficient Self-Evolving LLM Agent via Contextual Information Density Maximization (V1.0)cited by 10
  5. 2026HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agentscited by 10
  6. 2026Agentic Memory Enhanced Recursive Reasoning for Root Cause Localization in Microservicescited by 9
  7. 2026Hypothesize-Then-Verify: Speculative Root Cause Analysis for Microservices with Pathwise Parallelismcited by 8
  8. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  9. 2026Co-Evolving Skill Generation and Policy Optimizationcited by 7
  10. 2025Experience-Evolving Multi-Turn Tool-Use Agent with Hybrid Episodic-Procedural Memorycited by 6
  11. 2026SeeUPO: Sequence-Level Agentic-RL with Convergence Guaranteescited by 6
  12. 2026SAGE: Multi-Agent Self-Evolution for LLM Reasoningcited by 6
  13. 2026AgenticGEO: A Self-Evolving Agentic System for Generative Engine Optimizationcited by 6
  14. 2026COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflowscited by 6
  15. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  16. 2026SwiftMem: Fast Agentic Memory via Query-aware Indexingcited by 5
  17. 2026Large Language Model Agents Are Not Always Faithful Self-Evolverscited by 5
  18. 2026Agent-Omit: Adaptive Context Omission for Efficient LLM Agentscited by 5
  19. 2026Complementary RL: Towards Efficient Experience-Driven Agent Learningcited by 5
  20. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  21. 2026DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Trainingcited by 5
  22. 2026E2E-REME: Towards End-to-End Microservices Auto-Remediation via Experience-Simulation Reinforcement Fine-Tuningcited by 4
  23. 2026Learning Agentic Policy from Action Guidancecited by 4
  24. 2026APPO: Agentic Procedural Policy Optimizationcited by 4
  25. 2026Self-Verification Dilemma: Experience-Driven Suppression of Overused Checking in LLM Reasoningcited by 3
  26. 2026PREPING: Zero-Task Agent Memory Construction via Self-Practicecited by 3
  27. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 3
  28. 2026Self-evolving Embodied AIcited by 2
  29. 2026Code2Math: Can Your Code Agent Effectively Evolve Math Problems Through Exploration?cited by 2
  30. 2026El Agente Forjador: Task-Driven Agent Generation for Quantum Simulationcited by 2
  31. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directionscited by 2
  32. 2026MAGE: Multi-Agent Self-Evolution with Co-Evolutionary Knowledge Graphscited by 2
  33. 2026MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoningcited by 2
  34. 2026RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agentscited by 2
  35. 2026Escaping the Self-Confirmation Trap: An Execute-Distill-Verify Paradigm for Agentic Experience Learningcited by 2
  36. 2026Learning Stateful Predictive Knowledge From Experiencecited by 2
  37. 2026State2State: Environment-Derived Mid-Training for LLM Agentscited by 2
  38. 2026MetaRSI / RSI2: A Meta-Recursive Self-Improving System for Recursive Self-Improving Systems Themselvescited by 2

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  3. 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolution
  4. 2025Evolving in Tasks: Empowering the Multi-modality Large Language Model as the Computer Use Agent
  5. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems
  6. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  7. 2026Evolving-RL: End-to-End Optimization of Experience-Driven Self-Evolving Capability within Agents
  8. 2025MemEvolve: Meta-Evolution of Agent Memory Systems