mlsanityData to 14 Oct 2026

VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation

arXiv 2606.06819 ↗First posted 5 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reasoning Video Object Segmentation (RVOS) demands a sophisticated integration of temporal dynamics, spatial details, and linguistic reasoning to achieve precise pixel-level localization. Existing methods are limited to reasoning over fixed initial inputs and lack the capacity to actively acquire further visual evidence, which is often essential for resolving complex references in long or intricate videos. To address this, we propose VideoSEG-O3, the first multi-turn reinforcement learning framework for RVOS that emulates the human textit``coarse-to-fine'' cognitive process. It employs a textitmulti-turn temporal-spatial chain-of-thought to capture fine-grained details by iteratively pinpointing critical intervals and keyframes. Additionally, to enable the policy to perceive segmentation quality beyond mere text probability of texttt[SEG] during the RL stage, we introduce textitSEG-aware logit calibration, which integrates pixel-wise segmentation feedback directly into the token-level logits. Furthermore, we design a textitdecoupled thinking trace to hierarchically decompose the reasoning process into temporal, spatial, and linguistic dimensions, and construct VTS-CoT, a specialized cold-start dataset featuring comprehensive reasoning trajectories. The code and models will be released at https://github.com/Dmmm1997/VideoSEG-O3.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 54 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

49 of 54 references.

References · 49

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  4. 2016Modeling Context in Referring Expressionscited by 1,063
  5. 2023VILA: On Pre-training for Visual Language Modelscited by 692
  6. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  7. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  8. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
  9. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
  10. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  11. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  12. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
37 more
  1. 2023Self-Chained Image-Language Model for Video Localization and Question Answeringcited by 203
  2. 2022Language as Queries for Referring Video Object Segmentationcited by 182
  3. 2018Video Object Segmentation with Language Referring Expressionscited by 162
  4. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  5. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  6. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  7. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  8. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
  9. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  10. 2023PG-Video-LLaVA: Pixel Grounding Large Video-Language Modelscited by 51
  11. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  12. 2025TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMscited by 50
  13. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  14. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 44
  15. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
  16. 2025GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentationcited by 34
  17. 2023Tracking with Human-Intent Reasoningcited by 33
  18. 2024HyperSeg: Towards Universal Visual Segmentation with Large Language Modelcited by 33
  19. 2024InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Modelscited by 31
  20. 2025SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learningcited by 31
  21. 2025ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundationscited by 30
  22. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  23. 2025UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoningcited by 28
  24. 2025The Devil is in Temporal Token: High Quality Video Reasoning Segmentationcited by 26
  25. 2024An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Groundingcited by 21
  26. 2024Motion-Grounded Video Reasoning: Understanding and Perceiving Motion at Pixel Levelcited by 19
  27. 2025LENS: Learning to Segment Anything with Unified Reinforced Reasoningcited by 18
  28. 2025Reinforcing Video Reasoning Segmentation to Think Before It Segmentscited by 15
  29. 2024Temporally Consistent Referring Video Object Segmentation with Hybrid Memorycited by 14
  30. 2025Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmarkcited by 11
  31. 2025PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discriminationcited by 10
  32. 2025CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videoscited by 9
  33. 2025CoPRS: Learning Positional Prior from Chain-of-Thought for Reasoning Segmentationcited by 9
  34. 2025Long-RVOS: A Comprehensive Benchmark for Long-term Referring Video Object Segmentationcited by 8
  35. 2025Improving Generalized Visual Grounding with Instance-aware Joint Learningcited by 8
  36. 2025VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learningcited by 6
  37. 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation
  2. 2026VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation
  3. 2025FrameMind: Frame-Interleaved Video Reasoning via Reinforcement Learning
  4. 2026Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding
  5. 2025Open-o3-Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence
  6. 2026Training-Free Spatio-temporal Decoupled Reasoning Video Segmentation with Adaptive Object Memory
  7. 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning
  8. 2025Reasoning Segmentation for Images and Videos: A Survey