EnvHarness: Awakening Static Worlds for Agent Learning
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 51 of its 62 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
51 of 62 references.
Closest ancestors · 4
Same direction, newest first
- 2025Don't Just Fine-tune the Agent, Tune the Environment11 Oct
- 2024EnvGen: Generating and Adapting Environments via LLMs for Training Embodied Agents18 Mar
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Models25 May
- 2023Reflexion: Language Agents with Verbal Reinforcement Learning20 Mar
Other references · 47
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen3 Technical Reportcited by 8,820
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 876
- 2024Self-Rewarding Language Modelscited by 681
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
35 more
- 2024Agent Workflow Memorycited by 383
- 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 353
- 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
- 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
- 2025ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memorycited by 284
- 2026SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learningcited by 278
- 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
- 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
- 2020Prioritized Level Replaycited by 198
- 2019Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutionscited by 192
- 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 183
- 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 148
- 2024SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulationcited by 111
- 2025Agent0: Unleashing Self-Evolving Agents from Zero Data via Tool-Integrated Reasoningcited by 75
- 2026Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learningcited by 66
- 2026EnvScaler: Scaling Tool-Interactive Environments for LLM Agent via Programmatic Synthesiscited by 63
- 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
- 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 47
- 2025GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulatorscited by 42
- 2026MetaClaw: Just Talk -- An Agent That Meta-Learns and Evolves in the Wildcited by 37
- 2025InSTA: Towards Internet-Scale Training For Agentscited by 32
- 2026Qwen-AgentWorld: Language World Models for General Agentscited by 28
- 2026SWE-World: Building Software Engineering Agents in Docker-Free Environmentscited by 25
- 2026From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skillscited by 22
- 2024Refining Minimax Regret for Unsupervised Environment Designcited by 18
- 2025LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Trainingcited by 18
- 2026G-Zero: Self-Play for Open-Ended Generation from Zero Datacited by 17
- 2025Environment Scaling for Interactive Agentic Experience Collection: A Surveycited by 13
- 2026SEAL: Synergistic Co-Evolution of Agents and Learning Environmentscited by 12
- 2026SPADE: Self-Play in Adaptive Synthetic Executable Environmentscited by 12
- 2026Safe and Scalable Web Agent Learning via Recreated Websitescited by 11
- 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
- 2026Simulate to Generalize: Scaling Stateful Supervision for API-calling Agents using LLM World Modelscited by 2
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 18
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnessescited by 13
- 2026EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learningcited by 1
- 2026Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcutscited by 1
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnessescited by 1
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
- 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
- 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?cited by 0
- 2026SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecastingcited by 0
- 2026Questioning the Questions: Sustaining Self-Evolution in Reasoning Modelscited by 0
- 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?cited by 0
- 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvementcited by 0
- 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents14 Oct 2026
- 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement10 Oct 2026
- 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?6 Oct 2026
- 2026Questioning the Questions: Sustaining Self-Evolution in Reasoning Models3 Oct 2026
- 2026SEER: Self-Evolving Event Reasoning and Retrieval for Time Series Forecasting2 Oct 2026
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnesses2 Oct 2026
- 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?28 Sep 2026
- 2026CompoWorld: Compositional Environment Scaling for General Agents27 Sep 2026
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement24 Sep 2026
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnesses21 Sep 2026
- 2026Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts16 Sep 2026
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worlds14 Sep 2026
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnessescited by 13
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnessescited by 1
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvementcited by 0
- 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?cited by 0
- 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?cited by 0
- 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agentscited by 0
- 2026From Failures to Learnable Worlds: Policy-Conditioned Environment Generation for Navigation Agents14 Oct 2026
- 2026ServeLearnBench: How Well Can Agents Self-Improve from Serving Experience?6 Oct 2026
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnesses2 Oct 2026
- 2026AutoDataBench: Can Agents Write the Data That Feeds the Self-Improvement Loop?28 Sep 2026
- 2026Breaking the Environment Wall: Evolving LLM Agent Environments for Recursive Self-Improvement24 Sep 2026
- 2026RRSI: Regularized Recursive Self-Improvement of Agent Harnesses21 Sep 2026
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worldscited by 18
- 2026Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcutscited by 1
- 2026CompoWorld: Compositional Environment Scaling for General Agentscited by 0
- 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvementcited by 0
- 2026From Existing Methods to New Research Ideas: Two Perspectives on Recursive Self-Improvement10 Oct 2026
- 2026CompoWorld: Compositional Environment Scaling for General Agents27 Sep 2026
- 2026Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts16 Sep 2026
- 2026Dream-RSI: Recursive Self-Improvement through Evolving Worlds14 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026MemoHarness: Agent Harnesses That Learn from Experience
- 2026EnvCraft: Synthesizing Executable Environments in Agentic RL for Claw-like Agent
- 2026EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
- 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents
- 2026EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL
- 2026EnvACE: Internalizing Environment Dynamics via World Rehearsal for Agentic Reinforcement Learning
- 2026EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
- 2026Harness-Zero: Harness Distillation via Agent-as-Harness