mlsanityData to 14 Oct 2026

Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

arXiv 2609.11318 ↗First posted 10 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchmarks mainly evaluate medium-horizon exploration and rarely test whether agents can sustain long, dependency-heavy research processes. We introduce Mr. LHDR (Multimodal real-world Long-Horizon Deep Research), a benchmark for evaluating real-world deep research over long, irreducible chains of interdependent evidence across eight categories. Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions with a mean dependency depth of 10.4 before reaching a short, unique, and verifiable answer. Questions incorporate multimodal evidence, including images, maps, PDFs, logos, charts, tables, and video frames, with at least one non-text element that changes the reasoning state. Mr. LHDR evaluates both final answers and the correctness of intermediate conclusions under annotated dependencies. We evaluate general models, deep research systems, and agent frameworks using Overall Accuracy (OA), Strict Accuracy (SA), Checklist Score (CS), and Dependency-Aware Checklist Score (DACS). Results show that even the strongest system achieves only 43.1% OA and 34.3% SA, indicating that final-answer accuracy substantially overestimates complete research success. Removing images reduces DACS by 12.6 points, demonstrating the importance of multimodal evidence, while SA consistently declines as reasoning chains become longer. These findings reveal sustained, dependency-consistent evidence integration, rather than isolated fact retrieval, as a key bottleneck for current deep research agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 42 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

38 of 42 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains1 Mar
  2. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents28 Jan
  3. 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents18 Jan
  4. 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking23 Oct
  5. 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL12 Sep
  6. 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
  7. 2025RAVine: Reality-Aligned Evaluation for Agentic Search22 Jul
  8. 2025WebSailor: Navigating Super-human Reasoning for Web Agent3 Jul
  9. 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents23 Jun
  10. 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
  11. 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
  12. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
3 more
  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agents25 Jul
  2. 2023Mind2Web: Towards a Generalist Agent for the Web9 Jun
  3. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents4 Jul

Other references · 23

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  8. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  10. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  11. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  12. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
11 more
  1. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  2. 2025Seed1.5-VL Technical Reportcited by 373
  3. 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
  4. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  5. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
  6. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
  7. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agentscited by 38
  8. 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 36
  9. 2025Benchmarking Multimodal Knowledge Conflict for Large Multimodal Modelscited by 19
  10. 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understandingcited by 9
  11. 2025Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmarkcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome
  2. 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents
  3. 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
  4. 2025Deep Research: A Survey of Autonomous Research Agents
  5. 2026EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools
  6. 2026Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
  7. 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents
  8. 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents