mlsanityData to 14 Oct 2026

OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents

arXiv 2605.05185 ↗First posted 6 May 2026cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Deep search has become a crucial capability for frontier multimodal agents, enabling models to solve complex questions through active search, evidence verification, and multi-step reasoning. Despite rapid progress, top-tier multimodal search agents remain difficult to reproduce, largely due to the absence of open high-quality training data, transparent trajectory synthesis pipelines, or detailed training recipes. To this end, we introduce OpenSearch-VL, a fully open-source recipe for training frontier multimodal deep search agents with agentic reinforcement learning. First, we curated a dedicated pipeline to construct high-quality training data through Wikipedia path sampling, fuzzy entity rewriting, and source-anchor visual grounding, which jointly reduce shortcuts and one-step retrieval collapse. Based on this pipeline, we curate two training datasets, SearchVL-SFT-36k for SFT and SearchVL-RL-8k for RL. Besides, we design a diverse tool environment that unifies text search, image search, OCR, cropping, sharpening, super-resolution, and perspective correction, enabling agents to combine active perception with external knowledge acquisition. Finally, we propose a multi-turn fatal-aware GRPO training algorithm that handles cascading tool failures by masking post-failure tokens while preserving useful pre-failure reasoning through one-sided advantage clamping. Built on this recipe, OpenSearch-VL delivers substantial performance gains, with over 10-point average improvements across seven benchmarks, and achieves results comparable to proprietary commercial models on several tasks. We will release all data, code, and models to support open research on multimodal deep search agents.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 41 of its 49 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

41 of 49 references.

References · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  9. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  10. 2024A Survey on LLM-as-a-Judgecited by 1,423
  11. 2025OpenAI GPT-5 System Cardcited by 989
  12. 2025Video-R1: Reinforcing Video Reasoning in MLLMscited by 484
29 more
  1. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 245
  2. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  3. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  4. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  5. 2016FVQA: Fact-based Visual Question Answeringcited by 181
  6. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
  7. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
  8. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  9. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66
  10. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  11. 2025Visual Agentic Reinforcement Fine-Tuningcited by 55
  12. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 53
  13. 2025DeepMMSearch-R1: Empowering Multimodal LLMs in Multimodal Web Searchcited by 42
  14. 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 37
  15. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 34
  16. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learningcited by 33
  17. 2026Gen-Searcher: Reinforcing Agentic Search for Image Generationcited by 30
  18. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 27
  19. 2025PALADIN: Self-Correcting Language Model Agents to Cure Tool-Failure Casescited by 22
  20. 2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 20
  21. 2025ARES: Multimodal Adaptive Reasoning via Difficulty-Aware Token-Level Entropy Shapingcited by 17
  22. 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesiscited by 17
  23. 2025On Group Relative Policy Optimization Collapse in Agent Search: The Lazy Likelihood-Displacementcited by 15
  24. 2026Exploring Reasoning Reward Model for Agentscited by 15
  25. 2025Perception in Reflectioncited by 13
  26. 2026AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoningcited by 7
  27. 2026OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Taskscited by 2
  28. 2025CoRPO: Adding a Correctness Bias to GRPO Improves Generalizationcited by 1
  29. 2026Fix Before Search: Benchmarking Agentic Query Visual Pre-processing in Multimodal Retrieval-augmented Generationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 21.

21 papers.

Sort:

By area:

  1. 2026Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agentscited by 8
  2. 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoningcited by 8
  3. 2026PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignmentcited by 6
  4. 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulationcited by 5
  5. 2026From Web to Pixels: Bringing Agentic Search into Visual Perceptioncited by 3
  6. 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 3
  7. 2026SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Searchcited by 2
  8. 2026DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agentscited by 2
  9. 2026Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agentcited by 2
  10. 2026VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End?cited by 2
  11. 2026WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agentscited by 1
  12. 2026VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understandingcited by 0
9 more
  1. 2026VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrievalcited by 0
  2. 2026Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agentscited by 0
  3. 2026AdaVDR: Adaptive Tool Use and Reflection for Video Deep Researchcited by 0
  4. 2026LiteSearch-VL: Small Multimodal Search Agents via Trajectory Distillation and Synthetic Step-DPOcited by 0
  5. 2026Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratificationcited by 0
  6. 2026Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agentscited by 0
  7. 2026OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Videocited by 0
  8. 2026Neos: Learning to Interleave Search and Visual Reasoningcited by 0
  9. 2026ActICL: Post-Training for Agentic Multimodal In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards
  2. 2026VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learning
  3. 2026Towards Long-horizon Agentic Multimodal Search
  4. 2026OpenSeeker-v2: Pushing the Limits of Search Agents with Informative and High-Difficulty Trajectories
  5. 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management
  6. 2026InterLV-Search: Benchmarking Interleaved Multimodal Agentic Search
  7. 2026MTA-Agent: An Open Recipe for Multimodal Deep Search Agents
  8. 2026OpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data