# GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation

arXiv: https://arxiv.org/abs/2504.07962 · HTML: https://mlsanity.com/paper/2504.07962/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 9 of its 54 references that are in the map.
- First posted on arXiv 10 Apr 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 9 of 9)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memory](https://mlsanity.com/paper/2411.11922/) (2024)
- [PG-Video-LLaVA: Pixel Grounding Large Video-Language Models](https://mlsanity.com/paper/2311.13435/) (2023)
- [VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos](https://mlsanity.com/paper/2411.04923/) (2024)
- [Tracking with Human-Intent Reasoning](https://mlsanity.com/paper/2312.17448/) (2023)
- [ViLLa: Video Reasoning Segmentation with Large Language Model](https://mlsanity.com/paper/2407.14500/) (2024)

## Built on by (most cited first, 30 of 34)

- [UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning](https://mlsanity.com/paper/2509.18094/) (2025)
- [Advancing Complex Video Object Segmentation via Progressive Concept Construction](https://mlsanity.com/paper/2507.15852/) (2025)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [Object-centric Video Question Answering with Visual Grounding and Referring](https://mlsanity.com/paper/2507.19599/) (2025)
- [Deforming Videos to Masks: Flow Matching for Referring Video Segmentation](https://mlsanity.com/paper/2510.06139/) (2025)
- [CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videos](https://mlsanity.com/paper/2505.18561/) (2025)
- [Refer-Agent: A Collaborative Multi-Agent System with Reasoning and Reflection for Referring Video Object Segmentation](https://mlsanity.com/paper/2602.03595/) (2026)
- [LMMs Meet Object-Centric Vision: Understanding, Segmentation, Editing and Generation](https://mlsanity.com/paper/2604.11789/) (2026)
- [VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understanding](https://mlsanity.com/paper/2601.07290/) (2026)
- [BrokenVideos: A Benchmark Dataset for Fine-Grained Artifact Localization in AI-Generated Videos](https://mlsanity.com/paper/2506.20103/) (2025)
- [MomentSeg: Moment-Centric Sampling for Enhanced Video Pixel Understanding](https://mlsanity.com/paper/2510.09274/) (2025)
- [Annotations as Rollouts: Efficient and Scalable Reinforcement Learning for Video MLLMs](https://mlsanity.com/paper/2608.20492/) (2026)
- [Point Linguist Model: Segment Any Object via Bridged Large 3D-Language Model](https://mlsanity.com/paper/2509.07825/) (2025)
- [Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation](https://mlsanity.com/paper/2510.19592/) (2025)
- [UFVideo: Towards Unified Fine-Grained Video Cooperative Understanding with Large Language Models](https://mlsanity.com/paper/2512.11336/) (2025)
- [SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs](https://mlsanity.com/paper/2603.12382/) (2026)
- [SetCon: Towards Open-Ended Referring Segmentation via Set-Level Concept Prediction](https://mlsanity.com/paper/2605.20110/) (2026)
- [Evidence-Backed Video Question Answering](https://mlsanity.com/paper/2607.11862/) (2026)
- [SVAC: Scaling Is All You Need For Referring Video Object Segmentation](https://mlsanity.com/paper/2509.24109/) (2025)
- [CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation](https://mlsanity.com/paper/2511.17755/) (2025)
- [Detector-Empowered Video Large Language Model for Efficient Spatio-Temporal Grounding](https://mlsanity.com/paper/2512.06673/) (2025)
- [STORM: End-to-End Referring Multi-Object Tracking in Videos](https://mlsanity.com/paper/2604.10527/) (2026)
- [Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification](https://mlsanity.com/paper/2605.17531/) (2026)
- [Temporal Grounding as a Learning Signal for Referring Video Object Segmentation](https://mlsanity.com/paper/2508.11955/) (2025)
- [Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentation](https://mlsanity.com/paper/2509.05751/) (2025)
- [Online Reasoning Video Object Segmentation](https://mlsanity.com/paper/2604.11411/) (2026)
- [Weakly-Supervised Referring Video Object Segmentation through Text Supervision](https://mlsanity.com/paper/2604.17797/) (2026)
- [VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation](https://mlsanity.com/paper/2606.06819/) (2026)
- [Boosting Text-Driven Video Segmentation via Geometry-Aware Distillation](https://mlsanity.com/paper/2606.24464/) (2026)
- [FeVOS: Foresight Expression Video Object Segmentation](https://mlsanity.com/paper/2606.25585/) (2026)

## Abstract (arXiv)

This paper proposes a novel framework utilizing multi-modal large language models (MLLMs) for referring video object segmentation (RefVOS). Previous MLLM-based methods commonly struggle with the dilemma between "Ref" and "VOS": they either specialize in understanding a few key frames (global reasoning) or tracking objects on continuous frames (local reasoning), and rely on external VOS or frame selectors to mitigate the other end of the challenge. However, our framework GLUS shows that global and local consistency can be unified into a single video segmentation MLLM: a set of sparse "context frames" provides global information, while a stream of continuous "query frames" conducts local object tracking. This is further supported by jointly training the MLLM with a pre-trained VOS memory bank to simultaneously digest short-range and long-range temporal information. To improve the information efficiency within the limited context window of MLLMs, we introduce object contrastive learning to distinguish hard false-positive objects and a self-refined framework to identify crucial frames and perform propagation. By collectively integrating these insights, our GLUS delivers a simple yet effective baseline, achieving new state-of-the-art for MLLMs on the MeViS and Ref-Youtube-VOS benchmark. Our project page is at https://glus-video.github.io/.