mlsanityData to 7 Oct 2026

Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning

arXiv 2602.10090 ↗First posted 10 Feb 2026cs.AI cs.CL cs.LGcited by 51

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Recent advances in large language model (LLM) have empowered autonomous agents to perform multi-turn interactions with tools and environments. However, scaling such agent training is limited by the lack of diverse and reliable environments. In this paper, we propose Agent World Model (AWM), a fully synthetic environment generation pipeline. Using this pipeline, we scale to 1,000 environments covering everyday scenarios, in which agents can interact with rich toolsets and obtain high-quality observations. Notably, these environments are code-driven and backed by databases, providing more reliable and consistent state transitions than environments simulated by LLMs. Moreover, they enable more efficient agent interaction compared with collecting trajectories from realistic environments. To demonstrate the effectiveness of this resource, we perform large-scale reinforcement learning for multi-turn tool-use agents. Thanks to the fully executable environments and accessible database states, we can also design reliable reward functions. Experiments on three benchmarks show that training exclusively in synthetic environments, rather than benchmark-specific ones, yields strong out-of-distribution generalization. The code is available at https://github.com/Snowflake-Labs/agent-world-model.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 38 of its 71 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

38 of 71 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers20 Aug
  2. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment19 Feb

Other references · 36

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  12. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
24 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  2. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  3. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  4. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 755
  5. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  6. 2025Agentic Context Engineering: Evolving Contexts for Self-Improving Language Modelscited by 349
  7. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  8. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 218
  9. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  10. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 158
  11. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  12. 2024Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Modelscited by 97
  13. 2024AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectoriescited by 56
  14. 2025Simulating Environments with Reasoning Models for Agent Trainingcited by 52
  15. 2025Towards General Agentic Intelligence via Environment Scalingcited by 51
  16. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  17. 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 46
  18. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 41
  19. 2025Scaling Agent Learning via Experience Synthesiscited by 40
  20. 2025MCPToolBench++: A Large Scale AI Agent Model Context Protocol MCP Tool Use Benchmarkcited by 30
  21. 2025Procedural Environment Generation for Tool-Use Agentscited by 21
  22. 2025SynthAgent: Adapting Web Agents with Synthetic Supervisioncited by 12
  23. 2025AgentFly: Extensible and Scalable Reinforcement Learning for LM Agentscited by 8
  24. 2025TheMCPCompany: Creating General-purpose Agents with Task-specific Toolscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 51.

51 papers.

Sort:

By area:

  1. 2026Code as Agent Harnesscited by 40
  2. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
  3. 2026Gym-Anything: Turn any Software into an Agent Environmentcited by 24
  4. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  5. 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RLcited by 11
  6. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  7. 2026OpenComputer: Verifiable Software Worlds for Computer-Use Agentscited by 9
  8. 2026Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learningcited by 8
  9. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  10. 2026AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Securitycited by 7
  11. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 7
  12. 2026ClawEnvKit: Automatic Environment Generation for Claw-Like Agentscited by 6
38 more
  1. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  2. 2026NexForge: Scaling Agent Capabilities through Requirement-Driven Task Synthesis for LLMscited by 4
  3. 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agentscited by 3
  4. 2026TRACE: Capability-Targeted Agentic Trainingcited by 2
  5. 2026RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agentscited by 2
  6. 2026Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learningcited by 2
  7. 2026EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learningcited by 2
  8. 2026One Success Isn't Reliability: Thinkingbox, a Sandbox and Benchmark for Agents in Stateful Business Workflowscited by 2
  9. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 2
  10. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  11. 2026AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectoriescited by 2
  12. 2026Democratizing Tool Learning with Environments Fully Simulated by a Free 8B Language Modelcited by 1
  13. 2026MCP-Cosmos: World Model-Augmented Agents for Complex Task Execution in MCP Environmentscited by 1
  14. 2026PhoneWorld: From Real-App Trajectories to Dynamic and Verifiable Environments for Phone-Use Agentscited by 1
  15. 2026A First-Principles Derivation of LLM Policy Optimization: From Expected Reward to GRPO and Its Structural Extensionscited by 1
  16. 2026OPD-Evolver: Cultivating Holistic Agent Evolver via On-Policy Distillationcited by 1
  17. 2026Masked Diffusion Language Models are Strong and Steerable Text-Based World Models for Agentic RLcited by 1
  18. 2026EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agentcited by 1
  19. 2026CurateEvo: Data-Curation Evolving for Agentic Post-Trainingcited by 0
  20. 2026AgentOmnia: Scaling Agentic Models for Full-Scenario Applicationscited by 0
  21. 2026Quo Vadis, World Modeling?cited by 0
  22. 2026From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRLcited by 0
  23. 2026MidTool: Mid-training Data Synthesis for Agentic Tool Usecited by 0
  24. 2026AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scalecited by 0
  25. 2026MCP-Universe RL: A Framework for Training MCP Tool-Use Agents via Reinforcement Learningcited by 0
  26. 2026Agent Seer: Synthesizing Scenarios from Specification Understandingcited by 0
  27. 2026Ignorance or Incompetence? Constructing Knowledge-Gated, Verifiable Tasks for LLM Agentscited by 0
  28. 2026AutoGym: Blueprint-First Generation of Verifiable Agent Gymscited by 0
  29. 2026Dual-Frontier: When Can an Agent Trust Its World Model?cited by 0
  30. 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modelingcited by 0
  31. 2026Verifiable Hidden Dynamics Play: Generating Agentic RL Environments from Solved Mechanismscited by 0
  32. 2026Agent-Editing World Model: Rethinking World Modeling for LLM Agentscited by 0
  33. 2026Rufus-Air: An Open LLM Post-Training Recipecited by 0
  34. 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
  35. 2026CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement Learningcited by 0
  36. 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
  37. 2026WEFT: Scaling Tool-Use Post-Training for General-Purpose Agentscited by 0
  38. 2026SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generationcited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?
  2. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  3. 2025A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning
  4. 2025Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
  5. 2026Reinforcement World Model Learning for LLM-based Agents
  6. 2026Mock Worlds, Real Skills: Building Small Agentic Language Models with Synthetic Tasks, Simulated Environments, and Rubric-Based Rewards
  7. 2026From Traces to Agentic Worlds: Agentic Language World Models for Interactive Environment Simulation
  8. 2026World Models as an Intermediary between Agents and the Real World