# Unleashing Hierarchical Reasoning: An LLM-Driven Framework for Training-Free Referring Video Object Segmentation

arXiv: https://arxiv.org/abs/2509.05751 · HTML: https://mlsanity.com/paper/2509.05751/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 11 of its 37 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 11 of 11)

- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [VISA: Reasoning Video Object Segmentation via Large Language Models](https://mlsanity.com/paper/2407.11325/) (2024)
- [GLUS: Global-Local Reasoning Unified into A Single Large Language Model for Video Segmentation](https://mlsanity.com/paper/2504.07962/) (2025)
- [Tracking with Human-Intent Reasoning](https://mlsanity.com/paper/2312.17448/) (2023)
- [Large Language Models for Spatial Trajectory Patterns Mining](https://mlsanity.com/paper/2310.04942/) (2023)
- [LangCoop: Collaborative Driving with Language](https://mlsanity.com/paper/2504.13406/) (2025)
- [ViLLa: Video Reasoning Segmentation with Large Language Model](https://mlsanity.com/paper/2407.14500/) (2024)
- [In-Context Learning State Vector with Inner and Momentum Optimization](https://mlsanity.com/paper/2404.11225/) (2024)
- [Automated Vehicles Should be Connected with Natural Language](https://mlsanity.com/paper/2507.01059/) (2025)
- [Referring Video Object Segmentation via Language-aligned Track Selection](https://mlsanity.com/paper/2412.01136/) (2024)

## Abstract (arXiv)

Referring Video Object Segmentation (RVOS) aims to segment an object of interest throughout a video based on a language description. The prominent challenge lies in aligning static text with dynamic visual content, particularly when objects exhibiting similar appearances with inconsistent motion and poses. However, current methods often rely on a holistic visual-language fusion that struggles with complex, compositional descriptions. In this paper, we propose PARSE-VOS, a novel, training-free framework powered by Large Language Models (LLMs), for a hierarchical, coarse-to-fine reasoning across text and video domains. Our approach begins by parsing the natural language query into structured semantic commands. Next, we introduce a spatio-temporal grounding module that generates all candidate trajectories for all potential target objects, guided by the parsed semantics. Finally, a hierarchical identification module select the correct target through a two-stage reasoning process: it first performs coarse-grained motion reasoning with an LLM to narrow down candidates; if ambiguity remains, a fine-grained pose verification stage is conditionally triggered to disambiguate. The final output is an accurate segmentation mask for the target object. PARSE-VOS achieved state-of-the-art performance on three major benchmarks: Ref-YouTube-VOS, Ref-DAVIS17, and MeViS.