mlsanityData to 7 Oct 2026

GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators

arXiv 2512.19682 ↗First posted 22 Dec 2025cs.CLcited by 34

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Training capable Large Language Model (LLM) agents is critically bottlenecked by the high cost and static nature of real-world interaction data. We address this by introducing GenEnv, a framework that establishes a difficulty-aligned co-evolutionary game between an agent and a scalable, generative environment simulator. Unlike traditional methods that evolve models on static datasets, GenEnv instantiates a dataevolving: the simulator acts as a dynamic curriculum policy, continuously generating tasks specifically tailored to the agent's ``zone of proximal development''. This process is guided by a simple but effective α-Curriculum Reward, which aligns task difficulty with the agent's current capabilities. We evaluate GenEnv on five benchmarks, including API-Bank, ALFWorld, BFCL, Bamboogle, and TravelPlanner. Across these tasks, GenEnv improves agent performance by up to +40.3% over 7B baselines and matches or exceeds the average performance of larger models. Compared to Gemini 2.5 Pro-based offline data augmentation, GenEnv achieves better performance while using 3.3× less data. By shifting from static supervision to adaptive simulation, GenEnv provides a data-efficient pathway for scaling agent capabilities.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 40 of its 49 references that are in the map. First posted on arXiv 22 Dec 2025.

Builds on

40 of 49 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Latent Collaboration in Multi-Agent Systems25 Nov
  2. 2025Scaling Synthetic Task Generation for Agents via Exploration29 Sep
  3. 2025LIMI: Less is More for Agency22 Sep
  4. 2025LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback2 Jun
  5. 2025Training LLM-Based Agents with Synthetic Self-Reflected Trajectories and Partial Masking26 May
  6. 2025Agent-R: Training Language Model Agents to Reflect via Iterative Self-Training20 Jan
  7. 2024AgentBank: Towards Generalized LLM Agents via Fine-Tuning on 50000+ Interaction Trajectories10 Oct
  8. 2024KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents5 Mar
  9. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb

Other references · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2024Qwen2.5 Technical Reportcited by 4,713
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2015Trust Region Policy Optimizationcited by 2,503
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  12. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
19 more
  1. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  2. 2025AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Societycited by 180
  3. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  4. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
  5. 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agentscited by 82
  6. 2025Agent Learning via Early Experiencecited by 79
  7. 2025Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agentscited by 68
  8. 2025Co-Evolving LLM Coder and Unit Tester via Reinforcement Learningcited by 51
  9. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  10. 2025STeCa: Step-level Trajectory Calibration for LLM Agent Learningcited by 37
  11. 2025Demystifying Reinforcement Learning in Agentic Reasoningcited by 32
  12. 2024Data Augmentation using Large Language Models: Data Perspectives, Learning Paradigms and Challengescited by 30
  13. 2025Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Rewardcited by 25
  14. 2025Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPOcited by 18
  15. 2025LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Trainingcited by 17
  16. 2024LLM-DA: Data Augmentation via Large Language Models for Few-Shot Named Entity Recognitioncited by 11
  17. 2025ComPO: Preference Alignment via Comparison Oraclescited by 11
  18. 2025On the Role of Preference Variance in Preference Optimizationcited by 3
  19. 2024Good Data Is All Imitation Learning Needscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 34.

34 papers.

Sort:

By area:

  1. 2026SkillX: Automatically Constructing Skill Knowledge Bases for Agentscited by 69
  2. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 35
  3. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  4. 2026RLAnything: Forge Environment, Policy, and Reward Model in Completely Dynamic RL Systemcited by 19
  5. 2026From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanismscited by 19
  6. 2026Learning to Continually Learn via Meta-learning Agentic Memory Designscited by 16
  7. 2026Self-Play Only Evolves When Self-Synthetic Pipeline Ensures Learnable Information Gaincited by 16
  8. 2026SimWorld Studio: Automatic Environment Generation with Evolving Coding Agent for Embodied Agent Learningcited by 13
  9. 2026Reward-free Alignment for Conflicting Objectivescited by 10
  10. 2026OVD: On-policy Verbal Distillationcited by 9
  11. 2026CoVerRL: Breaking the Consensus Trap in Label-Free Reasoning via Generator-Verifier Co-Evolutioncited by 9
  12. 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 9
22 more
  1. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  2. 2026SCALER:Synthetic Scalable Adaptive Learning Environment for Reasoningcited by 7
  3. 2026AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Securitycited by 7
  4. 2026EnvHarness: Awakening Static Worlds for Agent Learningcited by 7
  5. 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 6
  6. 2026Agent-Omit: Adaptive Context Omission for Efficient LLM Agentscited by 5
  7. 2026Self-Evolving World Models for LLM Agent Planningcited by 5
  8. 2026The Last AI Built by Humans: Toward Genuine Recursive Self-Improvementcited by 5
  9. 2026Discovery Foundation Models: Toward Open-Ended Discovery Intelligencecited by 3
  10. 2026AQuA: Recursively Self-Improving Quantitative Trading Research Agentscited by 2
  11. 2026What Makes Good Agentic Data? An ACE Lens on Data Generation for LLM Agentscited by 2
  12. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  13. 2026Environment Evolution for Terminal Agentscited by 2
  14. 2026Co-Evolution of Policy and Internal Reward for Language Agentscited by 1
  15. 2026Scalable Environments Drive Generalizable Agentscited by 1
  16. 2026Self-Modifying Lean Proof Agents with Verifier-Grounded Benchmark Coevolutioncited by 1
  17. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1
  18. 2026AgentOmnia: Scaling Agentic Models for Full-Scenario Applicationscited by 0
  19. 2026STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolutioncited by 0
  20. 2026World State Generatorcited by 0
  21. 2026SkillSandbox: Skill Verification via Dynamic Scenario Synthesiscited by 0
  22. 2026VICO: Visual Environments Co-Evolving for Vision-Language Model Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentGym: Evolving Large Language Model-based Agents across Diverse Environments
  2. 2025Test-Time Adaptation for LLM Agents via Environment Interaction
  3. 2026Position: Agentic Evolution is the Path to Evolving LLMs
  4. 2025Nex-N1: Agentic Models Trained via a Unified Ecosystem for Large-Scale Environment Construction
  5. 2026HERA: Harness-Environment Co-Evolution for Reliable Agentic Abstention
  6. 2025AutoEnv: Automated Environments for Measuring Cross-Environment Agent Learning
  7. 2024A Survey on Self-Evolution of Large Language Models
  8. 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning