# OfficeQA Pro: An Enterprise Benchmark for End-to-End Grounded Reasoning

arXiv: https://arxiv.org/abs/2603.08655 · HTML: https://mlsanity.com/paper/2603.08655/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 8 of its 13 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 8 of 8)

- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks](https://mlsanity.com/paper/2412.15204/) (2024)
- [FinanceBench: A New Benchmark for Financial Question Answering](https://mlsanity.com/paper/2311.11944/) (2023)
- [BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent](https://mlsanity.com/paper/2508.06600/) (2025)
- [ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems](https://mlsanity.com/paper/2505.11831/) (2025)
- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://mlsanity.com/paper/2407.12883/) (2024)
- [GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks](https://mlsanity.com/paper/2510.04374/) (2025)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)

## Built on by (most cited first, 30 of 39)

- [SkillOpt: Executive Strategy for Self-Evolving Agent Skills](https://mlsanity.com/paper/2605.23904/) (2026)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies](https://mlsanity.com/paper/2605.03596/) (2026)
- [MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution](https://mlsanity.com/paper/2607.05297/) (2026)
- [SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe](https://mlsanity.com/paper/2607.03451/) (2026)
- [Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model](https://mlsanity.com/paper/2607.22083/) (2026)
- [HarnessEvolve: Learning from Reference Trajectories for Reliable Agent Self-Evolution](https://mlsanity.com/paper/2609.00829/) (2026)
- [ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?](https://mlsanity.com/paper/2608.03874/) (2026)
- [Designing Benchmarks for Knowledge Work](https://mlsanity.com/paper/2605.23262/) (2026)
- [AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning](https://mlsanity.com/paper/2606.24526/) (2026)
- [Rethinking Self-Evolving Agents: Do We Still Need Prescribed Optimization Pipelines?](https://mlsanity.com/paper/2608.09629/) (2026)
- [SkillAdam: Stable and Efficient Skill Evolution for Agents](https://mlsanity.com/paper/2609.08944/) (2026)
- [Occamy-1.0: Open Pareto-frontier 35B Intelligence for Co-work](https://mlsanity.com/paper/2609.11977/) (2026)
- [BlueFin: Benchmarking LLM Agents on Financial Spreadsheets](https://mlsanity.com/paper/2605.30907/) (2026)
- [StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows](https://mlsanity.com/paper/2608.17800/) (2026)
- [IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier](https://mlsanity.com/paper/2609.10494/) (2026)
- [COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization](https://mlsanity.com/paper/2609.11682/) (2026)
- [Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts](https://mlsanity.com/paper/2609.18366/) (2026)
- [MiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement](https://mlsanity.com/paper/2610.11959/) (2026)
- [π2: Structure-Originated Reasoning Data Improves Long-Context Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2604.05114/) (2026)
- [Office Comprehension Benchmark](https://mlsanity.com/paper/2607.01245/) (2026)
- [Self-Supervised Skill Optimization](https://mlsanity.com/paper/2607.28777/) (2026)
- [Query-Only Backdoor Attacks on Self-Evolving Skills via Trajectory Poisoning](https://mlsanity.com/paper/2608.08303/) (2026)
- [Branch2Skill: Efficient Skill Evolution Through Reasoning Trees](https://mlsanity.com/paper/2608.08677/) (2026)
- [Temporal Misgrounding in Legal RAG: A Versioned-Corpus Benchmark for French Tax Law](https://mlsanity.com/paper/2608.09393/) (2026)
- [StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents](https://mlsanity.com/paper/2608.18050/) (2026)
- [Agentic Context Cracking: Token-Efficient Data Reasoning Agents via Adaptive Structuring of Unstructured Data](https://mlsanity.com/paper/2608.31082/) (2026)
- [Studying Without a Syllabus: Task-Agnostic Environment Preprocessing](https://mlsanity.com/paper/2609.10824/) (2026)
- [WinSyn: An Automated Pipeline for Realistic Enterprise Question-Answering Evaluation](https://mlsanity.com/paper/2609.12171/) (2026)

## Abstract (arXiv)

We introduce OfficeQA Pro, a benchmark for evaluating AI agents on grounded, multi-document reasoning over a large and heterogeneous document corpus. The corpus consists of U.S. Treasury Bulletins spanning nearly 100 years, comprising 89,000 pages and over 26 million numerical values. OfficeQA Pro consists of 133 questions that require precise document parsing, retrieval, and analytical reasoning across both unstructured text and tabular data. Frontier LLMs including Claude Opus 4.6, GPT-5.4, and Gemini 3.1 Pro Preview achieve less than 5% accuracy on OfficeQA Pro when relying on parametric knowledge, and less than 12% with additional access to the web. When provided directly with the document corpus, frontier agents still struggle on over half of questions, scoring 34.1% on average. We find that providing agents with a structured document representation produced by Databricks' ai_parse_document yields a 16.1% average relative performance gain across agents. We conduct additional ablations to study the effects of model selection, table representation, retrieval strategy, and test-time scaling on performance. Despite these improvements, significant headroom remains before agents can be considered reliable at enterprise-grade grounded reasoning.