mlsanityData to 7 Oct 2026

AgentGym: Evolving Large Language Model-based Agents across Diverse Environments

arXiv 2406.04151 ↗First posted 6 Jun 2024cs.AI cs.CLcited by 120

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Building generalist agents that can handle diverse tasks and evolve themselves across different environments is a long-term goal in the AI community. Large language models (LLMs) are considered a promising foundation to build such agents due to their generalized capabilities. Current approaches either have LLM-based agents imitate expert-provided trajectories step-by-step, requiring human supervision, which is hard to scale and limits environmental exploration; or they let agents explore and learn in isolated environments, resulting in specialist agents with limited generalization. In this paper, we take the first step towards building generally-capable LLM-based agents with self-evolution ability. We identify a trinity of ingredients: 1) diverse environments for agent exploration and learning, 2) a trajectory set to equip agents with basic capabilities and prior knowledge, and 3) an effective and scalable evolution method. We propose AgentGym, a new framework featuring a variety of environments and tasks for broad, real-time, uni-format, and concurrent agent exploration. AgentGym also includes a database with expanded instructions, a benchmark suite, and high-quality trajectories across environments. Next, we propose a novel method, AgentEvol, to investigate the potential of agent self-evolution beyond previously seen data across tasks and environments. Experimental results show that the evolved agents can achieve results comparable to SOTA models. We release the AgentGym suite, including the platform, dataset, benchmark, checkpoints, and algorithm implementations. The AgentGym suite is available on https://github.com/WooooDyy/AgentGym.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 52 of its 69 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

52 of 69 references.

Closest ancestors · 13

Same direction, newest first

  1. 2024ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomy21 Mar
  2. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models19 Mar
  3. 2024AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning23 Feb
  4. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents24 Jan
  5. 2023Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning22 Dec
  6. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMs19 Oct
  7. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct
  8. 2023FireAct: Toward Language Agent Fine-tuning9 Oct
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Survey14 Sep
  10. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  11. 2023BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents11 Aug
  12. 2023AgentBench: Evaluating LLMs as Agents7 Aug
1 more
  1. 2022A Generalist Agent12 May

Other references · 39

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  10. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  11. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  12. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
27 more
  1. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  2. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  3. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  4. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  5. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  6. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  7. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  8. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 417
  9. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 325
  10. 2018Maximum a Posteriori Policy Optimisationcited by 318
  11. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
  12. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  13. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  14. 2024V-STaR: Training Verifiers for Self-Taught Reasonerscited by 214
  15. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  16. 2024Trial and Error: Exploration-Based Trajectory Optimization for LLM Agentscited by 194
  17. 2023ADaPT: As-Needed Decomposition and Planning with Language Modelscited by 172
  18. 2024Secrets of RLHF in Large Language Models Part II: Reward Modelingcited by 170
  19. 2024Orca-Math: Unlocking the potential of SLMs in Grade School Mathcited by 151
  20. 2024AutoWebGLM: A Large Language Model-based Web Navigating Agentcited by 130
  21. 2024Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizingcited by 123
  22. 2024A Survey on Self-Evolution of Large Language Modelscited by 86
  23. 2023LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Modelscited by 84
  24. 2023ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agentcited by 78
  25. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
  26. 2023Making Large Language Models Better Reasoners with Alignmentcited by 63
  27. 2024Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learningcited by 63

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 59; 2026: 48.

120 papers.

Sort:

By area:

  1. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  2. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  3. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  4. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 291
  5. 2025Memory in the Age of AI Agentscited by 285
  6. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
  7. 2025Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluationcited by 83
  8. 2025Agent Learning via Early Experiencecited by 79
  9. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Trainingcited by 74
  10. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 69
  11. 2025AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making through Multi-Turn Reinforcement Learningcited by 68
  12. 2025ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoningcited by 57
38 more
  1. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  2. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuningcited by 51
  3. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  4. 2024Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usagecited by 49
  5. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimizationcited by 45
  6. 2025A Survey on the Optimization of Large Language Model-based Agentscited by 45
  7. 2025ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestrationcited by 44
  8. 2025Reasoning Language Models: A Blueprintcited by 41
  9. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?cited by 40
  10. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  11. 2024ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systemscited by 30
  12. 2025ATLaS: Agent Tuning via Learning Critical Stepscited by 29
  13. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  14. 2025ToolMind Technical Report: A Large-Scale, Reasoning-Enhanced Tool-Use Datasetcited by 27
  15. 2025AgentRM: Enhancing Agent Generalization with Reward Modelingcited by 25
  16. 2026The Long-Horizon Task Mirage? Diagnosing Where and Why Agentic Systems Breakcited by 25
  17. 2025SWE-Dev: Building Software Engineering Agents with Training and Inference Scalingcited by 24
  18. 2024Can We Rely on LLM Agents to Draft Long-Horizon Plans? Let's Take TravelPlanner as an Examplecited by 22
  19. 2025DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environmentscited by 22
  20. 2025SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugscited by 22
  21. 2025MedAgentGym: A Scalable Agentic Training Environment for Code-Centric Reasoning in Biomedical Data Sciencecited by 22
  22. 2024Unlocking the Future: Exploring Look-Ahead Planning Mechanistic Interpretability in Large Language Modelscited by 21
  23. 2025The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvementcited by 19
  24. 2024Very Large-Scale Multi-Agent Simulation in AgentScopecited by 18
  25. 2025SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligencecited by 17
  26. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  27. 2025Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Maskingcited by 16
  28. 2025EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learningcited by 15
  29. 2026daVinci-Dev: Agent-native Mid-training for Software Engineeringcited by 15
  30. 2025DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generationcited by 14
  31. 2025AgentSwift: Efficient LLM Agent Design via Value-guided Hierarchical Searchcited by 14
  32. 2025Generalizable End-to-End Tool-Use RL with Synthetic CodeGymcited by 14
  33. 2025Plan-over-Graph: Towards Parallelable LLM Agent Schedulecited by 13
  34. 2025Ready Jurist One: Benchmarking Language Agents for Legal Intelligence in Dynamic Environmentscited by 13
  35. 2025PilotRL: Training Language Model Agents via Global Planning-Guided Progressive Reinforcement Learningcited by 11
  36. 2026Policy and World Modeling Co-Training for Language Agentscited by 11
  37. 2024Are LLMs Socially Adaptive? Contrasting Belief Evolution in Large Language Models and Humanscited by 10
  38. 2024Agent Skill Acquisition for Large Language Models via CycleQDcited by 10

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
  2. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges
  3. 2026AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments
  4. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects
  5. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence
  6. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
  7. 2025Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments
  8. 2024Offline Training of Language Model Agents with Functions as Learnable Weights