VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation
Not on the map. Why?
Abstract (arXiv)
Reasoning Video Object Segmentation (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning to achieve precise pixel-level localization. Existing methods are limited to reasoning over fixed initial inputs and lack the capacity to actively acquire further visual evidence, which is often essential for resolving complex references in long or intricate videos. To address this, we propose VideoSEG-O3, the first multi-turn reinforcement learning framework for RVOS that emulates the human textit``coarse-to-fine'' cognitive process. It employs a textitmulti-turn temporal-spatial chain-of-thought to capture fine-grained details by iteratively pinpointing critical intervals and keyframes. Additionally, to enable the policy to perceive segmentation quality beyond mere text probability of texttt[SEG] during the RL stage, we introduce textitSEG-aware logit calibration, which integrates pixel-wise segmentation feedback directly into the token-level logits. Furthermore, we design a textitdecoupled thinking trace to hierarchically decompose the reasoning process into temporal, spatial, and linguistic dimensions, and construct VTS-CoT, a specialized cold-start dataset featuring comprehensive reasoning trajectories. The code and models will be released at https://github.com/Dmmm1997/VideoSEG-O3.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 54 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
49 of 54 references.
References · 49
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
- 2016Modeling Context in Referring Expressionscited by 1,063
- 2023VILA: On Pre-training for Visual Language Modelscited by 692
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
- 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
- 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
- 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
- 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
37 more
- 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
- 2022Language as Queries for Referring Video Object Segmentationcited by 182
- 2018Video Object Segmentation with Language Referring Expressionscited by 162
- 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
- 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
- 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
- 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
- 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
- 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
- 2023PG-Video-LLaVA: Pixel Grounding Large Video-Language Modelscited by 51
- 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
- 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
- 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
- 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 44
- 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
- 2025GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentationcited by 34
- 2023Tracking with Human-Intent Reasoningcited by 33
- 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Modelcited by 33
- 2024InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Modelscited by 31
- 2025SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learningcited by 31
- 2025ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundationscited by 30
- 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
- 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 28
- 2025The Devil is in Temporal Token: High Quality Video Reasoning Segmentationcited by 26
- 2024An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Groundingcited by 21
- 2024Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Levelcited by 19
- 2025LENS: Learning to Segment Anything with Unified Reinforced Reasoningcited by 18
- 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
- 2024Temporally Consistent Referring Video Object Segmentation with Hybrid Memorycited by 14
- 2025Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmarkcited by 11
- 2025PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discriminationcited by 10
- 2025CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videoscited by 9
- 2025CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentationcited by 9
- 2025Long-RVOS: A Comprehensive Benchmark for Long-term Referring Video Object Segmentationcited by 8
- 2025Improving Generalized Visual Grounding with Instance-aware Joint Learningcited by 8
- 2025VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learningcited by 6
- 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation
- 2026VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation
- 2025FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
- 2026Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding
- 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence
- 2026Training-Free Spatio-temporal Decoupled Reasoning Video Segmentation with Adaptive Object Memory
- 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning
- 2025Reasoning Segmentation for Images and Videos: A Survey