Reinforcing Video Reasoning Segmentation to Think Before It Segments
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Video reasoning segmentation (VRS) endeavors to delineate referred objects in videos guided by implicit instructions that encapsulate human intent and temporal logic. Previous approaches leverage large vision language models (LVLMs) to encode object semantics into <SEG> tokens for mask prediction. However, this paradigm suffers from limited interpretability during inference and suboptimal performance due to inadequate spatiotemporal reasoning. Drawing inspiration from seminal breakthroughs in reinforcement learning, we introduce Veason-R1, a specialized LVLM for VRS that emphasizes structured reasoning in segmentation. Veason-R1 is trained through Group Relative Policy Optimization (GRPO) augmented with Chain-of-Thought (CoT) initialization. To begin with, we curate high-quality CoT training data to instill structured reasoning trajectories, bridging video-level semantics and frame-level spatial grounding, yielding the supervised fine-tuned model Veason-SFT. Subsequently, GRPO fine-tuning encourages efficient exploration of the reasoning space by optimizing reasoning chains. To this end, we incorporate a holistic reward mechanism that synergistically enhances spatial alignment and temporal consistency, bolstering keyframe localization and fine-grained grounding. Comprehensive empirical evaluations demonstrate that Veason-R1 achieves state-of-the-art performance on multiple benchmarks, surpassing prior art by significant margins (e.g., +1.3 J &F in ReVOS and +10.0 J &F in ReasonVOS), while exhibiting robustness to hallucinations (+8.8 R). Our code and model weights will be available at Veason-R1.
Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 30 of its 45 references that are in the map. First posted on arXiv 15 Aug 2025.
Builds on
30 of 45 references.
Closest ancestors · 15
Same direction, newest first
- 2025Improving the Reasoning of Multi-Image Grounding in MLLMs via Reinforcement Learning1 Jul
- 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learning27 Jun
- 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoning29 May
- 2025SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning28 May
- 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning20 May
- 2025MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO19 May
- 2025VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning17 May
- 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
3 more
Other references · 15
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2024OpenAI o1 System Cardcited by 1,244
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 413
- 2025Seed1.5-VL Technical Reportcited by 373
- 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
- 2025Time-R1: Post-Training Large Vision Language Model for Temporal Video Groundingcited by 146
3 more
- 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectivescited by 122
- 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 41
- 2024InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Modelscited by 31
Built on by
Citing papers per year: 2025: 3; 2026: 12.
15 papers.
Sort:
By area:
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twicecited by 21
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
- 2025Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Modelcited by 2
- 2025Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentationcited by 2
- 2026Evidence-Backed Video Question Answeringcited by 2
- 2026Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarificationcited by 1
- 2026AgentRVOS for MeViS-Text Track of 5th PVUW Challenge: 3rd Methodcited by 0
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentationcited by 0
- 2026SteerSeg: Attention Steering for Reasoning Video Segmentationcited by 0
- 2026MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentationcited by 0
- 2026Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thoughtcited by 0
- 2026VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentationcited by 0
3 more
- 2026Less from More: Reinforcing Sparse Video Reasoning from Dense References7 Oct 2026
- 2026Evidence-Backed Video Question Answering13 Jul 2026
- 2026Event-Aware Instructed Assistant for Referring Video Segmentation25 Jun 2026
- 2026FeVOS: Foresight Expression Video Object Segmentation24 Jun 2026
- 2026VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation5 Jun 2026
- 2026Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thought26 May 2026
- 2026MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation26 May 2026
- 2026Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification17 May 2026
- 2026SteerSeg: Attention Steering for Reasoning Video Segmentation14 May 2026
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation8 May 2026
- 2026AgentRVOS for MeViS-Text Track of 5th PVUW Challenge: 3rd Method20 Apr 2026
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice8 Jan 2026
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twicecited by 21
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
- 2025Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentationcited by 2
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentationcited by 0
- 2026SteerSeg: Attention Steering for Reasoning Video Segmentationcited by 0
- 2026Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thoughtcited by 0
- 2026FeVOS: Foresight Expression Video Object Segmentationcited by 0
- 2026Less from More: Reinforcing Sparse Video Reasoning from Dense Referencescited by 0
- 2026Less from More: Reinforcing Sparse Video Reasoning from Dense References7 Oct 2026
- 2026FeVOS: Foresight Expression Video Object Segmentation24 Jun 2026
- 2026Can Segmentation Models Understand the World? Towards Proactive Affordance Reasoning via Visual Chain-of-Thought26 May 2026
- 2026SteerSeg: Attention Steering for Reasoning Video Segmentation14 May 2026
- 2026RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation8 May 2026
- 2026VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice8 Jan 2026
- 2025Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation22 Oct 2025
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models6 Oct 2025
- 2025Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Modelcited by 2
- 2026Evidence-Backed Video Question Answeringcited by 2
- 2026Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarificationcited by 1
- 2026AgentRVOS for MeViS-Text Track of 5th PVUW Challenge: 3rd Methodcited by 0
- 2026MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentationcited by 0
- 2026VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentationcited by 0
- 2026Event-Aware Instructed Assistant for Referring Video Segmentationcited by 0
- 2026Evidence-Backed Video Question Answering13 Jul 2026
- 2026Event-Aware Instructed Assistant for Referring Video Segmentation25 Jun 2026
- 2026VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation5 Jun 2026
- 2026MedVol-R1: Reward-Driven Evidence Grounding for Volumetric Reasoning Segmentation26 May 2026
- 2026Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification17 May 2026
- 2026AgentRVOS for MeViS-Text Track of 5th PVUW Challenge: 3rd Method20 Apr 2026
- 2025Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model9 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025VideoSeg-R1:Reasoning Video Object Segmentation via Reinforcement Learning
- 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learning
- 2026VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation
- 2025Reinforcing Video Reasoning with Focused Thinking
- 2026VisionCoach: Reinforcing Grounded Video Reasoning via Visual-Perception Prompting
- 2025ViSS-R1: Self-Supervised Reinforcement Video Reasoning
- 2024VISA: Reasoning Video Object Segmentation via Large Language Models
- 2025Video-Thinker: Sparking "Thinking with Videos" via Reinforcement Learning