EntWorld: A Holistic Environment and Benchmark for Verifiable Enterprise GUI Agents
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Recent advances in Multimodal Large Language Models (MLLMs) have enabled agents to operate in open-ended web and operating system environments. However, existing benchmarks predominantly target consumer-oriented scenarios (e.g., e-commerce and travel booking), failing to capture the complexity and rigor of professional enterprise workflows. Enterprise systems pose distinct challenges, including high-density user interfaces, strict business logic constraints, and a strong reliance on precise, state-consistent information retrieval-settings in which current generalist agents often struggle. To address this gap, we introduce EntWorld, a large-scale benchmark consisting of 1,756 tasks across six representative enterprise domains, including customer relationship management (CRM), information technology infrastructure library (ITIL), and enterprise resource planning (ERP) systems. Unlike previous datasets that depend on fragile execution traces or extensive manual annotation, EntWorld adopts a schema-grounded task generation framework that directly reverse-engineers business logic from underlying database schemas, enabling the synthesis of realistic, long-horizon workflows. Moreover, we propose a SQL-based deterministic verification mechanism in building datasets that replaces ambiguous visual matching with rigorous state-transition validation. Experimental results demonstrate that state-of-the-art models (e.g., GPT-4.1) achieve 47.61% success rate on EntWorld, substantially lower than the human performance, highlighting a pronounced enterprise gap in current agentic capabilities and the necessity of developing domain-specific agents. We release EntWorld as a rigorous testbed to facilitate the development and evaluation of the next generation of enterprise-ready digital agents.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 29 of its 36 references that are in the map. First posted on arXiv 25 Jan 2026.
Builds on
29 of 36 references.
References · 29
- 2023GPT-4 Technical Reportcited by 18,396
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
- 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
17 more
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 438
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
- 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
- 2024Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMscited by 176
- 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 165
- 2025OS-Harm: A Benchmark for Measuring Safety of Computer Use Agentscited by 111
- 2024Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?cited by 56
- 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
- 2025OSUniverse: Benchmark for Multimodal GUI-navigation AI Agentscited by 10
- 2025GUI-360^circ: A Comprehensive Dataset and Benchmark for Computer-Using Agentscited by 10
- 2023An Empirical Study of NetOps Capability of Pre-Trained Large Language Modelscited by 9
- 2025The Digital Cybersecurity Expert: How Far Have We Come?cited by 7
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmarkcited by 5
- 2026Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflowscited by 2
- 2026EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessionscited by 2
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Softwarecited by 1
- 2026Running the Gauntlet: Challenging Agentic Taskscited by 0
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agentscited by 0
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software15 Sep 2026
- 2026DI-Bench: Systematically Generating In-Domain Data Intelligence Benchmarks for Enterprise Agents4 Sep 2026
- 2026EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions22 Jun 2026
- 2026Running the Gauntlet: Challenging Agentic Tasks12 Jun 2026
- 2026Beyond the All-in-One Agent: Benchmarking Role-Specialized Multi-Agent Collaboration in Enterprise Workflows9 May 2026
- 2026WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark13 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026World of Workflows: A Benchmark for Bringing World Models to Enterprise Systems
- 2026EnterpriseBench: Benchmarking LLM Agents on Enterprise-Level Strategic Reasoning and Decision-Making
- 2026E-Bench: Benchmarking Multi-Step Tool-Use Agents in Real-World Product Scenarios
- 2025Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments
- 2026PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows
- 2026Beyond Text-to-SQL: An Agentic LLM System for Governed Enterprise Analytics APIs
- 2026BC-Bench: Evaluating Agentic Engineering in a Domain-Specific Language for ERP
- 2026EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings