mlsanityData to 7 Oct 2026

From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents

arXiv 2603.25342 ↗First posted 26 Mar 2026cs.LGcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Deep Research Agents (DRAs) aim to answer complex questions by searching the web, checking evidence, and synthesizing conclusions across heterogeneous sources. We introduce a category-theoretic framework for evaluating and improving such agents. The framework treats deep research as a structured mapping from user intent to evidence-grounded conclusions, making retrieval traces, cross-source alignment, and final synthesis explicit. Guided by this view, we derive a mechanism-aware benchmark of 296 bilingual questions. The benchmark targets four structural skills central to real research: following multi-hop evidence chains, verifying claims across sources, re-ordering fragmented information, and rejecting unsupported assumptions. We evaluate 16 frontier systems with human verification and find that these structural tasks remain highly challenging: the best system reaches only 19.9% average accuracy. The results show that strong agents can sometimes reorganize evidence and detect false premises, but still struggle with long-horizon synthesis and intersection-heavy verification. Beyond evaluation, the same theory also leads to practical system improvements. We instantiate theory-guided interventions such as tracked search, which preserves retrieval traces, and category tools, which add explicit verification and synthesis steps. These interventions yield measurable gains in API-based deep research systems. Our work therefore provides both a challenging benchmark and concrete design guidance for building more reliable research agents.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 25 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

15 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
  2. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun

Other references · 13

  1. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  2. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 139
  3. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 86
  4. 2025Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Taskscited by 63
  5. 2019A synthetic approach to Markov kernels, conditional independence and theorems on sufficient statisticscited by 55
  6. 2017Backprop as Functor: A compositional perspective on supervised learningcited by 44
  7. 2022On the Power of Foundation Modelscited by 37
  8. 2021Category Theory in Machine Learningcited by 36
  9. 2021An enriched category theory of language: from syntax to semanticscited by 21
  10. 2023Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category Theorycited by 12
  11. 2022Unifying Causal Inference and Reinforcement Learning using Higher-Order Category Theorycited by 4
  12. 2019Characterizing the invariances of learning algorithms using category theorycited by 3
1 more
  1. 2022Categoroids: Universal Conditional Independencecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools
  2. 2026DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
  3. 2025Deep Research: A Survey of Autonomous Research Agents
  4. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
  5. 2026Argus: Evidence Assembly for Scalable Deep Research Agents
  6. 2026Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
  7. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
  8. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents