Delegation Intelligence in Deep Search: A Controllable Framework for Disentangled Capability Diagnosis
Not on the map. Why?
Abstract (arXiv)
Deep search is becoming a core capability of modern agent systems, yet it is typically evaluated solely based on end-to-end answer accuracy. This coupled evaluation paradigm entangles retrieval quality, long-context comprehension, evidence verification, and tool-use decisions, making it difficult to determine whether a model truly knows when and how to delegate information seeking to search. To this end: (1) We formalize this meta-capability as Delegation Intelligence in deep search and decompose it into complementary dimensions-Search Decision-Making (recognizing information insufficiency and deciding whether, when, and how to search) and Information Synthesis and Verification (aggregating evidence from multiple sources, judging source reliability, and synthesizing information under noisy, potentially adversarial conditions). (2) To enable disentangled and reproducible measurement, we develop a controllable synthesis pipeline built on document-grounded reverse engineering. This yields a general recipe for constructing controlled deep-search evaluations rather than a single fixed dataset. (3) As a concrete instantiation, we construct DelegSearchBench, together with a disentangled evaluation protocol that isolates each capability dimension by varying document composition and tool access. (4) Across representative models, we demonstrate that deep-search competence cannot be adequately characterized by final-answer accuracy alone...
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 26 of its 35 references that are in the map. First posted on arXiv 26 Jul 2026.
Builds on
26 of 35 references.
References · 26
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,804
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
- 2024A Survey on LLM-as-a-Judgecited by 1,399
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,266
- 2023GAIA: a benchmark for General AI Assistantscited by 1,102
- 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,063
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 902
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 628
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 483
- 2025Humanity's Last Examcited by 478
- 2025WebWalker: Benchmarking LLMs in Web Traversalcited by 173
14 more
- 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 159
- 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 140
- 2025Natural Emergent Misalignment from Reward Hacking in Production RLcited by 126
- 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 57
- 2025ReportBench: Evaluating Deep Research Agents via Academic Survey Taskscited by 36
- 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 32
- 2025MedBrowseComp: Benchmarking Medical Deep Research and Computer Usecited by 29
- 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Datacited by 29
- 2026Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflowscited by 11
- 2025Beyond Itinerary Planning-A Real-World Benchmark for Multi-Turn and Tool-Using Travel Taskscited by 10
- 2025TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesiscited by 4
- 2026DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domaincited by 3
- 2026SearchSwarm: Towards Delegation Intelligence in Agentic LLMs for Long-Horizon Deep Researchcited by 3
- 2024CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learningcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.