Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchmarks mainly evaluate medium-horizon exploration and rarely test whether agents can sustain long, dependency-heavy research processes. We introduce Mr. LHDR (Multimodal real-world Long-Horizon Deep Research), a benchmark for evaluating real-world deep research over long, irreducible chains of interdependent evidence across eight categories. Each question is constructed from a hidden Node-Relation graph and requires an average of 12.1 necessary intermediate conclusions with a mean dependency depth of 10.4 before reaching a short, unique, and verifiable answer. Questions incorporate multimodal evidence, including images, maps, PDFs, logos, charts, tables, and video frames, with at least one non-text element that changes the reasoning state. Mr. LHDR evaluates both final answers and the correctness of intermediate conclusions under annotated dependencies. We evaluate general models, deep research systems, and agent frameworks using Overall Accuracy (OA), Strict Accuracy (SA), Checklist Score (CS), and Dependency-Aware Checklist Score (DACS). Results show that even the strongest system achieves only 43.1% OA and 34.3% SA, indicating that final-answer accuracy substantially overestimates complete research success. Removing images reduces DACS by 12.6 points, demonstrating the importance of multimodal evidence, while SA consistently declines as reasoning chains become longer. These findings reveal sustained, dependency-consistent evidence integration, rather than isolated fact retrieval, as a key bottleneck for current deep research agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 42 references that are in the map. First posted on arXiv 10 Sep 2026.
Builds on
38 of 42 references.
Closest ancestors · 15
Same direction, newest first
- 2026MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains1 Mar
- 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agents28 Jan
- 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents18 Jan
- 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking23 Oct
- 2025DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL12 Sep
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agent8 Aug
- 2025RAVine: Reality-Aligned Evaluation for Agentic Search22 Jul
- 2025WebSailor: Navigating Super-human Reasoning for Web Agent3 Jul
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents23 Jun
- 2025DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents13 Jun
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Models9 Jan
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
3 more
Other references · 23
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
11 more
- 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
- 2025Seed1.5-VL Technical Reportcited by 373
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
- 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
- 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
- 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
- 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agentscited by 38
- 2025Benchmarking Retrieval-Augmented Multimodal Generation for Document Question Answeringcited by 36
- 2025Benchmarking Multimodal Knowledge Conflict for Large Multimodal Modelscited by 19
- 2025RAVENEA: A Benchmark for Multimodal Retrieval-Augmented Visual Culture Understandingcited by 9
- 2025Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmarkcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome
- 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents
- 2026From Intent to Evidence: A Categorical Approach for Structural Evaluation of Deep Research Agents
- 2025Deep Research: A Survey of Autonomous Research Agents
- 2026EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools
- 2026Struct-Searcher: Agentic Structural Thinking Advances Multimodal Deep Information Seeking
- 2025ADRA-Bank: A Modular Benchmark for Academic Deep Research Agents
- 2025WebResearcher: Unleashing unbounded reasoning capability in Long-Horizon Agents