# Seg-ReSearch: Segmentation with Interleaved Reasoning and External Search

arXiv: https://arxiv.org/abs/2602.04454 · HTML: https://mlsanity.com/paper/2602.04454/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 17 of its 52 references that are in the map.
- First posted on arXiv 4 Feb 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 17 of 17)

- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement](https://mlsanity.com/paper/2503.06520/) (2025)
- [DeepEyesV2: Toward Agentic Multimodal Model](https://mlsanity.com/paper/2511.05271/) (2025)
- [WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent](https://mlsanity.com/paper/2508.05748/) (2025)
- [AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress](https://mlsanity.com/paper/2511.08325/) (2025)
- [In-the-Flow Agentic System Optimization for Effective Planning and Tool Use](https://mlsanity.com/paper/2510.05592/) (2025)
- [OneThinker: All-in-one Reasoning Model for Image and Video](https://mlsanity.com/paper/2512.03043/) (2025)
- [Visual Agentic Reinforcement Fine-Tuning](https://mlsanity.com/paper/2505.14246/) (2025)
- [VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning](https://mlsanity.com/paper/2505.12081/) (2025)
- [Agent-R1: A Unified and Modular Framework for Agentic Reinforcement Learning](https://mlsanity.com/paper/2511.14460/) (2025)
- [Advancing Complex Video Object Segmentation via Progressive Concept Construction](https://mlsanity.com/paper/2507.15852/) (2025)
- [Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning](https://mlsanity.com/paper/2510.25992/) (2025)
- [Long-RVOS: A Comprehensive Benchmark for Long-term Referring Video Object Segmentation](https://mlsanity.com/paper/2505.12702/) (2025)

## Built on by (most cited first, 6 of 6)

- [From Web to Pixels: Bringing Agentic Search into Visual Perception](https://mlsanity.com/paper/2605.12497/) (2026)
- [StAR: Segment Anything Reasoner](https://mlsanity.com/paper/2603.14382/) (2026)
- [AnchorSeg: Language Grounded Query Banks for Reasoning Segmentation](https://mlsanity.com/paper/2604.18562/) (2026)
- [Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification](https://mlsanity.com/paper/2605.17531/) (2026)
- [Revisit to Segment: Working Memory Distillation for Reasoning Segmentation](https://mlsanity.com/paper/2609.34863/) (2026)
- [EviRover: Reinforcing Agentic Perception Beyond a Glance](https://mlsanity.com/paper/2609.40230/) (2026)

## Abstract (arXiv)

Segmentation based on language has been a popular topic in computer vision. While recent advances in multimodal large language models (MLLMs) have endowed segmentation systems with reasoning capabilities, these efforts remain confined by the frozen internal knowledge of MLLMs, which limits their potential for real-world scenarios that involve up-to-date information or domain-specific concepts. In this work, we propose Seg-ReSearch, a novel segmentation paradigm that overcomes the knowledge bottleneck of existing approaches. By enabling interleaved reasoning and external search, Seg-ReSearch empowers segmentation systems to handle dynamic, open-world queries that extend beyond the frozen knowledge of MLLMs. To effectively train this capability, we introduce a hierarchical reward design that harmonizes initial guidance with progressive incentives, mitigating the dilemma between sparse outcome signals and rigid step-wise supervision. For evaluation, we construct OK-VOS, a challenging benchmark that explicitly requires outside knowledge for video object segmentation. Experiments on OK-VOS and two existing reasoning segmentation benchmarks demonstrate that our Seg-ReSearch improves state-of-the-art approaches by a substantial margin. Code and data will be released at https://github.com/iSEE-Laboratory/Seg-ReSearch.