mlsanityData to 7 Oct 2026

DRBENCHER: Can Your Agent Identify the Entity, Retrieve Its Properties and Do the Math?

arXiv 2604.09251 ↗First posted 10 Apr 2026cs.AIcited by 1

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Deep research agents increasingly interleave web browsing with multi-step computation, yet existing benchmarks evaluate these capabilities in isolation, creating a blind spot in assessing real-world performance. We introduce DRBENCHER, a synthetic benchmark generator for questions that require both browsing and computation, synthesized answer-first from knowledge-graph chains with no seed passages or provenance text. It enforces four criteria: verifiability (gold answers are computed by executing parameterized code over knowledge-graph values), complexity (multi-hop entity identification, property retrieval, and domain-specific computation), difficulty (a two-stage verification cascade filters out questions solvable by the generating model), and diversity (a greedy max-min embedding filter maximizes coverage). These criteria are realized via a unified answer-first pipeline spanning five domains: biochemistry, financial, geophysical, security, and history. Human evaluation shows 76% validity, with 35% of errors due to outdated knowledge-graph entries, highlighting an inherent limitation of systems that reason over evolving data. Automatic evaluation shows that the strongest frontier model achieves only 20% answer accuracy. Compared to manually constructed benchmarks (BrowseComp+, MATH-500, GPQA), DRBENCHER achieves the highest semantic diversity. We release DrBencher pipeline and the human-verified benchmark data at https://github.com/IBM/DrBencher.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 50 references that are in the map. First posted on arXiv 10 Apr 2026.

Builds on

32 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scaling14 Nov
  2. 2025WebAggregator: Enhancing Compositional Reasoning Capabilities of Deep Research Agent Foundation Models16 Oct
  3. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Training1 Aug

Other references · 29

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  10. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
17 more
  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  2. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  3. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  4. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  5. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  6. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  7. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
  8. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  9. 2023Rethinking Benchmark and Contamination for Language Models with Rephrased Samplescited by 192
  10. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  11. 2025WebShaper: Agentically Data Synthesizing via Information-Seeking Formalizationcited by 118
  12. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  13. 2020KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Basecited by 76
  14. 2024MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmarkcited by 41
  15. 2024AutoBencher: Towards Declarative Benchmark Constructioncited by 40
  16. 2025Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Modelscited by 14
  17. 2025Granite Embedding R2 Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Dr. Bench: A Multidimensional Evaluation for Deep Research Agents, from Answers to Reports
  2. 2026AgentIR: Reasoning-Aware Retrieval for Deep Research Agents
  3. 2025ResearchRubrics: A Benchmark of Prompts and Rubrics For Evaluating Deep Research Agents
  4. 2025Improving and Evaluating Open Deep Research Agents
  5. 2026DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments
  6. 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
  7. 2026DR-Arena: an Automated Evaluation Framework for Deep Research Agents
  8. 2026DeepResearch-9K: A Challenging Benchmark Dataset of Deep-Research Agent