mlsanityData to 7 Oct 2026

Benchmarking Hybrid Deep Research Across Database Querying and Web Search

arXiv 2609.09410 ↗First posted 8 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

While autonomous agents have made significant strides in "deep research" by iteratively navigating the open web to synthesize information, real-world problem-solving is rarely confined to a single environment. Complex analytical tasks inherently require agents to weave together evidence from both ambiguous unstructured text (e.g., the open web) and highly precise structured data (e.g., relational databases). However, existing benchmarks evaluate these modalities in isolation, failing to capture the critical "handoff" - the ability to preserve constraints when moving evidence between systems. We introduce HybridDeepResearch, to our knowledge the first deep-research benchmark that requires both web search and SQL to form a complete, verifiable answer. The benchmark contains 380 tool-dependent tasks grounded in LiveSQLBench-Base-Lite databases and public web corpora, validated through automated checks and human review, and covering three reasoning patterns: SQL2S, S2SQL, and Parallel. Evaluations across proprietary and open-weight models under various agentic scaffolds reveal that even state-of-the-art models like GLM-5.2, Claude-Sonnet-4.6 and GPT-5 achieve only about 50-54% Pass@8 on the hard subset. Notably, results show that directional reasoning is substantially more difficult than parallel intersection, highlighting that bridging structured and unstructured information spaces without losing constraints remains a major open challenge for agentic systems. Code and datasets are publicly available at GitHub (https://github.com/Snowflake-AI-Research/HybridDeepResearch) and Hugging Face (https://huggingface.co/datasets/Snowflake/HybridDeepResearch).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 55 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

38 of 55 references.

References · 38

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  5. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  6. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  7. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  11. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  12. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
26 more
  1. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  2. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  3. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  4. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  5. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  6. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  7. 2020HybridQA: A Dataset of Multi-Hop Question Answering over Tabular and Textual Datacited by 263
  8. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agentscited by 236
  9. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 183
  10. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  11. 2020Open Question Answering over Tables and Textcited by 163
  12. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 155
  13. 2024Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Modelscited by 150
  14. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  15. 2023WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferencescited by 100
  16. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judgecited by 75
  17. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAGcited by 34
  18. 2025Characterizing Deep Research: A Benchmark and Formal Definitioncited by 29
  19. 2025MedBrowseComp: Benchmarking Medical Deep Research and Computer Usecited by 28
  20. 2021Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answeringcited by 25
  21. 2025Benchmarking Deep Search over Heterogeneous Enterprise Datacited by 23
  22. 2023CompMix: A Benchmark for Heterogeneous Question Answeringcited by 12
  23. 2026MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chainscited by 11
  24. 2026DataSTORM: Deep Research on Large-Scale Databases using Exploratory Data Analysis and Data Storytellingcited by 7
  25. 2025Demystifying deep search: a holistic evaluation with hint-free multi-hop questions and factorised metricscited by 4
  26. 2026A Benchmark for Deep Information Synthesiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Deep Research Bench: Evaluating AI Web Research Agents
  2. 2025DeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research
  3. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
  4. 2026Evaluating the Search Agent in a Parallel World
  5. 2026SAGE: Benchmarking and Improving Retrieval for Deep Research Agents
  6. 2025Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning
  7. 2025LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild
  8. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports