mlsanityData to 14 Oct 2026

WorkSurface-Bench: Benchmarking Enterprise Agents on Multi-Surface Knowledge Routing

arXiv 2607.25765 ↗First posted 28 Jul 2026cs.CL cs.DBcited by 0

Not on the map. Why?

Abstract (arXiv)

Enterprise agents often need to integrate heterogeneous knowledge sources: documents for narrative facts, tables for computation, and dependency graphs for file relationships. Existing benchmarks typically evaluate retrieval or tool use without distinguishing whether an agent first selects the appropriate knowledge sources. We introduce WorkSurface-Bench, a benchmark for evaluating this capability as surface routing. It contains 1,151 atomic tasks derived from persona-scoped Workspace-Bench-Lite workspaces, spanning document, table, graph, and cross-surface questions. Its reference answers are auditable: table answers are reproduced through executed DuckDB queries, document answers are grounded in verified text spans, and graph answers are traced to source dependency annotations. We evaluate four model backbones across six controlled agent settings, yielding 27,624 protocol-error-free trajectories. Under gold-constrained tool access, agents achieve 98.7-99.8 Route F1, while Answer remains only 56.1-75.3 percent, showing that correct surface selection is necessary but insufficient for task completion. Matched interventions further show that surface hints improve Answer for three of four models, whereas removing irrelevant tools primarily improves routing and efficiency. In an independent three-annotator audit, all 200 sampled tasks pass all six quality criteria by majority vote, with 192 receiving unanimous judgments on every criterion. We release the dataset, construction pipeline, scoring code, and agent harness at https://github.com/haolpku/WorkSurface-Bench.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

18 of 25 references.

References · 18

  1. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  2. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  3. 2018Datasheets for Datasetscited by 1,386
  4. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  5. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  6. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
  7. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  8. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
  9. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  10. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 326
  11. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 264
  12. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
6 more
  1. 2021MultiModalQA: Complex Question Answering over Text, Tables and Imagescited by 179
  2. 2020Open Question Answering over Tables and Textcited by 163
  3. 2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 117
  4. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  5. 2025mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphscited by 5
  6. 2025T2-RAGBench: Text-and-Table Benchmark for Evaluating Retrieval-Augmented Generationcited by 3

Built on by

Not cited yet.