mlsanityData to 14 Oct 2026

Neos: Learning to Interleave Search and Visual Reasoning

arXiv 2610.17043 ↗First posted 12 Oct 2026cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent-environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 80 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

51 of 80 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Visual-Seeker: Towards Visual-Native Multimodal Agentic Search via Active Visual Reasoning13 Jun
  2. 2026Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents11 May
  3. 2026ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards22 Apr
  4. 2026Walk the Talk: Bridging the Reasoning-Action Gap for Thinking with Images via Multimodal Agentic Policy Optimization8 Apr
  5. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Models2 Feb
  6. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning30 Dec
  7. 2025RSAgent: Learning to Reason and Act for Text-Guided Segmentation via Multi-Turn Tool Invocations30 Dec
  8. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents29 Aug
  9. 2025Thyme: Think Beyond Images15 Aug
  10. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May

Other references · 41

  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  2. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  3. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  4. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  5. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  6. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  7. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 335
  8. 2025DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environmentscited by 286
  9. 2025WebSailor: Navigating Super-human Reasoning for Web Agentcited by 229
  10. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 207
  11. 2025WebDancer: Towards Autonomous Information Seeking Agencycited by 188
  12. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 145
29 more
  1. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
  2. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 124
  3. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
  4. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 119
  5. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 101
  6. 2025MiroThinker: Pushing the Performance Boundaries of Open-Source Research Agents via Model, Context, and Interactive Scalingcited by 81
  7. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
  8. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 66
  9. 2026DeepSearchQA: Bridging the Comprehensiveness Gap for Deep Research Agentscited by 50
  10. 2026REDSearcher: A Scalable and Cost-Efficient Framework for Long-Horizon Search Agentscited by 49
  11. 2025MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agentscited by 42
  12. 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 37
  13. 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 21
  14. 2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 20
  15. 2026Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Trainingcited by 18
  16. 2024Vision Search Assistant: Empower Vision-Language Models as Multimodal Search Enginescited by 16
  17. 2026Marco DeepResearch: Unlocking Efficient Deep Research Agents via Verification-Centric Designcited by 16
  18. 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Modelscited by 14
  19. 2026VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agentscited by 13
  20. 2026Towards Long-horizon Agentic Multimodal Searchcited by 11
  21. 2026VSearcher: Long-Horizon Multimodal Search Agent via Reinforcement Learningcited by 9
  22. 2026BrowseComp-V3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agentscited by 8
  23. 2026SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulationcited by 5
  24. 2026ABSeeker: Training Long-Horizon Search Agents via Answer-Backtracked Credit Assignmentcited by 3
  25. 2026SimpleSearch-VL: A Simple Recipe for Multimodal Agentic Deep Searchcited by 2
  26. 2026DeepSearch-World: Self-Distillation for Deep Search Agents in a Verifiable Environmentcited by 2
  27. 2026DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agentscited by 2
  28. 2026CAFE: Self-Improving Search Agents Need Co-Evolving Feedbackcited by 2
  29. 2026UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseCompcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
  2. 2026HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
  3. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch
  4. 2026Eliciting Self-Verification in Multimodal Reasoning Agents with Reinforcement Learning
  5. 2026POINTS-Seeker: An Open Recipe for Multimodal Search Agents with Visual Memory Management
  6. 2026Securing the Floor and Raising the Ceiling: A Merging-based Paradigm for Multi-modal Search Agents
  7. 2026SPyCE: Skill-Policy Co-evolution for Multimodal Agents
  8. 2025InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search