# EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings

arXiv: https://arxiv.org/abs/2603.13594 · HTML: https://mlsanity.com/paper/2603.13594/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 13 Mar 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 16 of 16)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale](https://mlsanity.com/paper/2409.08264/) (2024)
- [UserBench: An Interactive Gym Environment for User-Centric Agents](https://mlsanity.com/paper/2507.22034/) (2025)
- [WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting](https://mlsanity.com/paper/2405.00823/) (2024)
- [CRMArena-Pro: Holistic Assessment of LLM Agents Across Diverse Business Scenarios and Interactions](https://mlsanity.com/paper/2505.18878/) (2025)
- [Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments](https://mlsanity.com/paper/2510.27287/) (2025)

## Built on by (most cited first, 15 of 15)

- [ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces](https://mlsanity.com/paper/2604.05172/) (2026)
- [From Question Answering to Task Completion: A Survey on Agent System and Harness Design](https://mlsanity.com/paper/2606.20683/) (2026)
- [Terminal Agents Suffice for Enterprise Automation](https://mlsanity.com/paper/2604.00073/) (2026)
- [π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows](https://mlsanity.com/paper/2605.14678/) (2026)
- [On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification](https://mlsanity.com/paper/2608.18066/) (2026)
- [EvoHarnessBench: Can Your Agents Keep Pace with an Evolving Harness?](https://mlsanity.com/paper/2609.04280/) (2026)
- [Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows](https://mlsanity.com/paper/2605.08761/) (2026)
- [Do Enterprise Systems Need Learned World Models? The Importance of Context to Infer Dynamics](https://mlsanity.com/paper/2605.12178/) (2026)
- [AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale](https://mlsanity.com/paper/2608.20634/) (2026)
- [StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments](https://mlsanity.com/paper/2608.24804/) (2026)
- [No Task Fails Every Time: Why One-Shot Audits Are Structurally Blind to Agent Damage](https://mlsanity.com/paper/2608.15286/) (2026)
- [Continual Enterprise World Model Discovery in Dynamic Systems](https://mlsanity.com/paper/2609.19551/) (2026)
- [WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks](https://mlsanity.com/paper/2609.23806/) (2026)
- [Trust and Task Completion in the World of Consumer AI Agents](https://mlsanity.com/paper/2609.33017/) (2026)
- [Confidence Reasoning Graphs: Structured Confidence Estimation for LLM Agents](https://mlsanity.com/paper/2610.07948/) (2026)

## Abstract (arXiv)

Large language models are shifting from passive information providers to active agents intended for complex workflows. However, their deployment as reliable AI workers in enterprise is stalled by benchmarks that fail to capture the intricacies of professional environments, specifically, the need for long-horizon planning amidst persistent state changes and strict access protocols. In this work, we introduce EnterpriseOps-Gym, a benchmark designed to evaluate agentic planning in realistic enterprise settings. Specifically, EnterpriseOps-Gym features a containerized sandbox with 164 database tables and 512 functional tools to mimic real-world search friction. Within this environment, agents are evaluated on 1,150 expert-curated tasks across eight mission-critical verticals (including Customer Service, HR, and IT). Our evaluation of 14 frontier models reveals critical limitations in state-of-the-art models: the top-performing Claude Opus 4.5 achieves only 37.4% success. Further analysis shows that providing oracle human plans improves performance by 14-35 percentage points, pinpointing strategic reasoning as the primary bottleneck. Additionally, agents frequently fail to refuse infeasible tasks (best model achieves 53.9%), leading to unintended and potentially harmful side effects. Our findings underscore that current agents are not yet ready for autonomous enterprise deployment. More broadly, EnterpriseOps-Gym provides a concrete testbed to advance the robustness of agentic planning in professional workflows.