Towards Long-horizon Agentic Multimodal Search
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Multimodal deep search agents have shown great potential in solving complex tasks by iteratively collecting textual and visual evidence. However, managing the heterogeneous information and high token costs associated with multimodal inputs over long horizons remains a critical challenge, as existing methods often suffer from context explosion or the loss of crucial visual signals. To address this, we propose a novel Long-horizon MultiModal deep search framework, named LMM-Searcher, centered on a file-based visual representation mechanism. By offloading visual assets to an external file system and mapping them to lightweight textual identifiers (UIDs), our approach mitigates context overhead while preserving multimodal information for future access. We equip the agent with a tailored fetch-image tool, enabling a progressive, on-demand visual loading strategy for active perception. Furthermore, we introduce a data synthesis pipeline designed to generate queries requiring complex cross-modal multi-hop reasoning. Using this pipeline, we distill 12K high-quality trajectories to fine-tune Qwen3-VL-Thinking-30A3B into a specialized multimodal deep search agent. Extensive experiments across four benchmarks demonstrate that our method successfully scales to 100-turn search horizons, achieving state-of-the-art performance among open-source models on challenging long-horizon benchmarks like MM-BrowseComp and MMSearch-Plus, while also exhibiting strong generalizability across different base models. Our code will be released in https://github.com/RUCAIBox/LMM-Searcher.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 56 of its 61 references that are in the map. First posted on arXiv 14 Apr 2026.
Builds on
56 of 61 references.
Closest ancestors · 6
Same direction, newest first
- 2026VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents17 Mar
- 2025DeepEyesV2: Toward Agentic Multimodal Model7 Nov
- 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents29 Aug
- 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents14 Aug
- 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent7 Aug
- 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents9 Nov
Other references · 50
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
- 2023Visual Instruction Tuningcited by 7,869
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023A Survey of Large Language Modelscited by 2,576
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
38 more
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023A Survey on Multimodal Large Language Modelscited by 909
- 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
- 2019Object Detection in 20 Years: A Surveycited by 531
- 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 452
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
- 2025Search-o1: Agentic Search-Enhanced Large Reasoning Modelscited by 433
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
- 2025R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learningcited by 336
- 2025BrowseComp-Plus: A More Fair and Transparent Evaluation Benchmark of Deep-Research Agentcited by 200
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
- 2016FVQA: Fact-based Visual Question Answeringcited by 181
- 2022A Survey of Vision-Language Pre-Trained Modelscited by 177
- 2025ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarizationcited by 141
- 2025A Survey of Context Engineering for Large Language Modelscited by 126
- 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
- 2018Scene Text Detection and Recognition: The Deep Learning Eracited by 114
- 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
- 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
- 2024Multimodal Alignment and Fusion: A Surveycited by 74
- 2025Virgo: A Preliminary Exploration on Reproducing o1-like MLLMcited by 68
- 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
- 2025SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesiscited by 55
- 2026Seed1.8 Model Card: Towards Generalized Real-World Agencycited by 50
- 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
- 2025Bring Reason to Vision: Understanding Perception and Reasoning through Model Mergingcited by 40
- 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
- 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 37
- 2026MiroThinker-1.7 & H1: Towards Heavy-Duty Research Agents via Verificationcited by 32
- 2023What Makes for Good Visual Instructions? Synthesizing Complex Visual Reasoning Instructions for Visual Instruction Tuningcited by 28
- 2025Large Vision-Language Model Alignment and Misalignment: A Survey Through the Lens of Explainabilitycited by 28
- 2026MiroFlow: Towards High-Performance and Robust Open-Source Agent Framework for General Deep Research Taskscited by 15
- 2024Exploring the Design Space of Visual Context Representation in Video MLLMscited by 6
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agentcited by 22
- 2026TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agentscited by 3
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 3
- 2026Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolutioncited by 2
- 2026SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Searchcited by 2
- 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Managementcited by 1
- 2026Search-to-World: Evaluation of 3D World Delivery from User Request through Web Searchcited by 1
- 2026Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agentscited by 0
- 2026EviRover: Reinforcing Agentic Perception Beyond a Glancecited by 0
- 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agentscited by 0
- 2026Neos: Learning to Interleave Search and Visual Reasoningcited by 0
- 2026Neos: Learning to Interleave Search and Visual Reasoning12 Oct 2026
- 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents4 Oct 2026
- 2026EviRover: Reinforcing Agentic Perception Beyond a Glance30 Sep 2026
- 2026Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents10 Sep 2026
- 2026Search-to-World: Evaluation of 3D World Delivery from User Request through Web Search7 Sep 2026
- 2026SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Search30 Jun 2026
- 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent29 Jun 2026
- 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearch2 Jun 2026
- 2026TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents11 May 2026
- 2026Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution9 May 2026
- 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management15 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents
- 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents
- 2026LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPO
- 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
- 2026ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards
- 2026DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Histories
- 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch