Neos: Learning to Interleave Search and Visual Reasoning
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent-environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 80 references that are in the map. First posted on arXiv 12 Oct 2026.
Builds on
51 of 80 references.
Closest ancestors · 10
Same direction, newest first
- 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning13 Jun
- 2026Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents11 May
- 2026ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards22 Apr
- 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization8 Apr
- 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models2 Feb
- 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning30 Dec
- 2025RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations30 Dec
- 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents29 Aug
- 2025Thyme: Think Beyond Images15 Aug
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
Other references · 41
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
- 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
- 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
- 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
- 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
- 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 145
29 more
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
- 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
- 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
- 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
- 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
- 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
- 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66
- 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 50
- 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
- 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
- 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 37
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 21
- 2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 20
- 2026Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Trainingcited by 18
- 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 16
- 2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 16
- 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Modelscited by 14
- 2026VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agentscited by 13
- 2026Towards Long-horizon Agentic Multimodal Searchcited by 11
- 2026VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learningcited by 9
- 2026BrowseComp-V3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agentscited by 8
- 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulationcited by 5
- 2026ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignmentcited by 3
- 2026SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Searchcited by 2
- 2026DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environmentcited by 2
- 2026DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agentscited by 2
- 2026CAFE: Self-Improving Search Agents Need Co-Evolving Feedbackcited by 2
- 2026UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseCompcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
- 2026HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
- 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch
- 2026Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning
- 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management
- 2026Securing the Floor and Raising the Ceiling: A Merging-based Paradigm for Multi-modal Search Agents
- 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agents
- 2025InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search