M3Searcher: Modular Multimodal Information Seeking Agency with Retrieval-Oriented Reasoning
Not on the map. Why?
Abstract (arXiv)
Recent advances in DeepResearch-style agents have demonstrated strong capabilities in autonomous information acquisition and synthesize from real-world web environments. However, existing approaches remain fundamentally limited to text modality. Extending autonomous information-seeking agents to multimodal settings introduces critical challenges: the specialization-generalization trade-off that emerges when training models for multimodal tool-use at scale, and the severe scarcity of training data capturing complex, multi-step multimodal search trajectories. To address these challenges, we propose M3Searcher, a modular multimodal information-seeking agent that explicitly decouples information acquisition from answer derivation. M3Searcher is optimized with a retrieval-oriented multi-objective reward that jointly encourages factual accuracy, reasoning soundness, and retrieval fidelity. In addition, we develop MMSearchVQA, a multimodal multi-hop dataset to support retrieval centric RL training. Experimental results demonstrate that M3Searcher outperforms existing approaches, exhibits strong transfer adaptability and effective reasoning in complex multimodal tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 14 Jan 2026.
Builds on
11 of 24 references.
References · 11
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 616
- 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
- 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
- 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 98
- 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 93
- 2024Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agentcited by 65
- 2025s3: You Don't Need That Much Data to Train a Search Agent via RLcited by 41
- 2025DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Searchcited by 41
Built on by
Citing papers per year: 2026: 3.
3 papers.