Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Environments serve as interactive systems for large language model (LLM) based agents across diverse scenarios and play a crucial role in driving the continual evolution of model capabilities. Despite this importance, existing work lacks a systematic categorization and deep analysis. This paper systematically studies current researches on agentic environments from the perspective of the environment engineering lifecycle, covering their modeling, synthesis, evaluation and application. Specifically, the paper first introduces representative environments from the perspectives of eight attributes and eight domains, providing detailed analyses of their development paths and highlighting their core capabilities. Second, for automated environment synthesis, two paradigms are introduced, such as symbolic synthesis and neural synthesis. This paper also shows different environment evaluation methods in each paradigm. Thirdly, the corresponding environment applications from the perspective of agent-environment co-evolution are discussed. In specific, the paper characterizes the primary pathways for agent evolution in dynamic environments from four complementary perspectives: memory-centric experience evolution, orchestration-centric workflow evolution, trajectory-centric offline evolution, and exploration-centric online evolution. And three paradigms of environment evolution are identified, namely neural-driven, difficulty-driven, and scaling-driven approaches. At last, several promising future directions are discussed, including Environment-as-a-Service, Multi-agent Environments, and Neural-Symbolic Environments.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 412 of its 582 references that are in the map. First posted on arXiv 10 Jun 2026.
Builds on
412 of 582 references.
Closest ancestors · 52
Same direction, newest first
- 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents7 Apr
- 2026ClawArena: Benchmarking AI Agents in Evolving Information Environments5 Apr
- 2026LOGIGEN: Logic-Driven Generation of Verifiable Agentic Tasks28 Feb
- 2026MagicAgent: Towards Generalized Agent Planning22 Feb
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning10 Feb
- 2026WideSeek-R1: Exploring Width Scaling for Broad Information Seeking via Multi-Agent Reinforcement Learning4 Feb
- 2026AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration3 Feb
- 2026IntentRL: Training Proactive User-intent Agents for Open-ended Deep Research via Reinforcement Learning3 Feb
- 2026LongCat-Flash-Thinking-2601 Technical Report23 Jan
- 2026Computer Environments Elicit General Agentic Intelligence in LLMs22 Jan
- 2026Toward Efficient Agents: Memory, Tool learning, and Planning20 Jan
- 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents18 Jan
38 more
- 2026AgencyBench: Benchmarking the Frontiers of Autonomous Agents in 1M-Token Real-World Contexts16 Jan
- 2026ACE-Router: Generalizing History-Aware Routing from MCP Tools to the Agent Web13 Jan
- 2025CASCADE: Cumulative Agentic Skill Creation through Autonomous Development and Evolution29 Dec
- 2025Agent2World: Learning to Generate Symbolic World Models via Adaptive Multi-Agent Feedback26 Dec
- 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators22 Dec
- 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skills18 Dec
- 2025Grammar Search for Multi-Agent Systems16 Dec
- 2025AgentEvolver: Towards Efficient Self-Evolving Agent System13 Nov
- 2025FLEX: Continuous Agent Evolution via Forward Learning from Experience9 Nov
- 2025Real-Time Reasoning Agents in Evolving Environments7 Nov
- 2025TPS-Bench: Evaluating AI Agents' Tool Planning & Scheduling Abilities in Compounding Tasks3 Nov
- 2025AgentFrontier: Expanding the Capability Frontier of LLM Agents with ZPD-Guided Data Synthesis28 Oct
- 2025DeepAgent: A General Reasoning Agent with Scalable Toolsets24 Oct
- 2025R-WoM: Retrieval-augmented World Model For Computer-use Agents13 Oct
- 2025Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Tasks9 Oct
- 2025Scaling Synthetic Task Generation for Agents via Exploration29 Sep
- 2025ARE: Scaling Up Agent Environments and Evaluations21 Sep
- 2025Towards General Agentic Intelligence via Environment Scaling16 Sep
- 2025Scaling Agents via Continual Pre-training16 Sep
- 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems10 Aug
- 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL6 Aug
- 2025General Agentic Planning Through Simulative Reasoning with World Models31 Jul
- 2025UserBench: An Interactive Gym Environment for User-Centric Agents29 Jul
- 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents17 Jun
- 2025TaskCraft: Automated Generation of Agentic Tasks11 Jun
- 2025World Modelling Improves Language Model Agents3 Jun
- 2025Self-Challenging Language Model Agents2 Jun
- 2025Dyna-Think: Synergizing Reasoning, Acting, and World Model Simulation in AI Agents31 May
- 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation29 May
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents29 May
- 2025Multi-Agent Collaboration via Evolving Orchestration26 May
- 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay4 Apr
- 2025OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning16 Feb
- 2024CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing22 Oct
- 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation1 Aug
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
- 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments6 Jun
- 2024Embodied LLM Agents Learn to Cooperate in Organized Teams19 Mar
50 newest of 52 All 52 (JSON)
Other references · 360
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2016OpenAI Gymcited by 1,736
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
38 more
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
- 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
- 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
- 2026World Action Models are Zero-shot Policiescited by 465
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
- 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
- 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 386
- 2024Agent Workflow Memorycited by 383
- 2024Measuring short-form factuality in large language modelscited by 373
- 2025ToolRL: Reward is All Tool Learning Needscited by 362
- 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
- 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
- 2026Causal World Modeling for Robot Controlcited by 321
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
- 2025Memory in the Age of AI Agentscited by 303
- 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 300
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
- 2024Diffusion Models Are Real-Time Game Enginescited by 296
- 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
- 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
- 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
- 2024TextGrad: Automatic "Differentiation" via Textcited by 273
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 248
50 most cited of 360 All 360 (JSON)
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 13
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 11
- 2026Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix Itcited by 7
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Docited by 4
- 2026The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillationcited by 3
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 3
- 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directionscited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agents27 Sep 2026
- 2026Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions21 Sep 2026
- 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement10 Sep 2026
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence21 Aug 2026
- 2026EnvHarness: Awakening Static Worlds for Agent Learning20 Aug 2026
- 2026The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation27 Jul 2026
- 2026Why Multi-Step Tool-Use Reinforcement Learning Collapses and How Supervisory Signals Fix It24 Jun 2026
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do21 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025Test-Time Adaptation for LLM Agents via Environment Interaction
- 2025Large Language Model Agent: A Survey on Methodology, Applications and Challenges
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligence
- 2026Position: Agentic Evolution is the Path to Evolving LLMs
- 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms
- 2026EvoArena: Tracking Memory Evolution for Robust LLM Agents in Dynamic Environments
- 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives
- 2026HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention