# ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations

arXiv: https://arxiv.org/abs/2501.14607 · HTML: https://mlsanity.com/paper/2501.14607/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 50 references that are in the map.
- First posted on arXiv 24 Jan 2025.

## Builds on (most cited first, 30 of 37)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Grounded Language-Image Pre-training](https://mlsanity.com/paper/2112.03857/) (2021)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation](https://mlsanity.com/paper/2206.02777/) (2022)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [Recognize Anything: A Strong Image Tagging Model](https://mlsanity.com/paper/2306.03514/) (2023)
- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [Language as Queries for Referring Video Object Segmentation](https://mlsanity.com/paper/2201.00487/) (2022)
- [Video Object Segmentation with Language Referring Expressions](https://mlsanity.com/paper/1803.08006/) (2018)
- [End-to-End Referring Video Object Segmentation with Multimodal Transformers](https://mlsanity.com/paper/2111.14821/) (2021)
- [Learning Two-Branch Neural Networks for Image-Text Matching Tasks](https://mlsanity.com/paper/1704.03470/) (2017)
- [DetGPT: Detect What You Need via Reasoning](https://mlsanity.com/paper/2305.14167/) (2023)
- [A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehension](https://mlsanity.com/paper/1909.07072/) (2019)
- [Modeling Relationships in Referential Expressions with Compositional Modular Networks](https://mlsanity.com/paper/1611.09978/) (2016)
- [Actor and Action Video Segmentation from a Sentence](https://mlsanity.com/paper/1803.07485/) (2018)
- [Learning to Compose and Reason with Language Tree Structures for Visual Grounding](https://mlsanity.com/paper/1906.01784/) (2019)
- [Zero-Shot Grounding of Objects from Natural Language Queries](https://mlsanity.com/paper/1908.07129/) (2019)
- [OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation](https://mlsanity.com/paper/2307.09356/) (2023)
- [SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentation](https://mlsanity.com/paper/2305.17011/) (2023)
- [Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentation](https://mlsanity.com/paper/2404.03645/) (2024)

## Built on by (most cited first, 30 of 30)

- [Spatia: Video Generation with Updatable Spatial Memory](https://mlsanity.com/paper/2512.15716/) (2025)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [Deforming Videos to Masks: Flow Matching for Referring Video Segmentation](https://mlsanity.com/paper/2510.06139/) (2025)
- [MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation](https://mlsanity.com/paper/2601.06874/) (2026)
- [Panoptic Captioning: An Equivalence Bridge for Image and Text](https://mlsanity.com/paper/2505.16334/) (2025)
- [PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of Complex Videos in the Wild](https://mlsanity.com/paper/2504.11326/) (2025)
- [GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling](https://mlsanity.com/paper/2606.20799/) (2026)
- [MomentSeg: Moment-Centric Sampling for Enhanced Video Pixel Understanding](https://mlsanity.com/paper/2510.09274/) (2025)
- [Motion-Aware Vision-Reference Alignment for Referring Multi-Object Tracking](https://mlsanity.com/paper/2511.17681/) (2025)
- [From Web to Pixels: Bringing Agentic Search into Visual Perception](https://mlsanity.com/paper/2605.12497/) (2026)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [ReferDINO-Plus: 2nd Solution for 4th PVUW MeViS Challenge at CVPR 2025](https://mlsanity.com/paper/2503.23509/) (2025)
- [Temporal-Conditional Referring Video Object Segmentation with Noise-Free Text-to-Video Diffusion Model](https://mlsanity.com/paper/2508.13584/) (2025)
- [Enhancing Sa2VA for Referent Video Object Segmentation: 2nd Solution for 7th LSVOS RVOS Track](https://mlsanity.com/paper/2509.15546/) (2025)
- [Segment-to-Act: Label-Noise-Robust Action-Prompted Video Segmentation Towards Embodied Intelligence](https://mlsanity.com/paper/2509.16677/) (2025)
- [TubeRMC: Tube-conditioned Reconstruction with Mutual Constraints for Weakly-supervised Spatio-Temporal Video Grounding](https://mlsanity.com/paper/2511.10241/) (2025)
- [VIRST: Video-Instructed Reasoning Assistant for SpatioTemporal Segmentation](https://mlsanity.com/paper/2603.27060/) (2026)
- [Don't Guess, Just Ask: Resolving Ambiguity in Referring Segmentation via Multi-turn Clarification](https://mlsanity.com/paper/2605.17531/) (2026)
- [G2TAM: Geometry Grounded Track Anything Model](https://mlsanity.com/paper/2607.03789/) (2026)
- [Context Consistency Learning via Sentence Removal for Semi-Supervised Video Paragraph Grounding](https://mlsanity.com/paper/2506.18476/) (2025)
- [CroBIM-V: Memory-Quality Controlled Remote Sensing Referring Video Object Segmentation](https://mlsanity.com/paper/2601.12076/) (2026)
- [Show Me When and Where: Towards Referring Video Object Segmentation in the Wild](https://mlsanity.com/paper/2603.14300/) (2026)
- [PinpointQA: A Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos](https://mlsanity.com/paper/2604.08991/) (2026)
- [Weakly-Supervised Referring Video Object Segmentation through Text Supervision](https://mlsanity.com/paper/2604.17797/) (2026)
- [VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation](https://mlsanity.com/paper/2606.06819/) (2026)
- [Boosting Text-Driven Video Segmentation via Geometry-Aware Distillation](https://mlsanity.com/paper/2606.24464/) (2026)
- [Segmenting, Fast and Slow: Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing](https://mlsanity.com/paper/2607.00124/) (2026)
- [ReflexTrack: A Feedback-Driven Agent for Training-Free Referring Video Object Segmentation](https://mlsanity.com/paper/2607.24098/) (2026)
- [CD-RMOT-Bench: Benchmarking the Cross-Domain Referring Multi-Object Tracking](https://mlsanity.com/paper/2607.25239/) (2026)
- [LD-RSVIS: A Large-Scale and Diverse Benchmark for Referring Surgical Video Instrument Segmentation](https://mlsanity.com/paper/2609.23067/) (2026)

## Abstract (arXiv)

Referring video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description. This is challenging as it involves deep vision-language understanding, pixel-level dense prediction and spatiotemporal reasoning. Despite notable progress in recent years, existing methods still exhibit a noticeable gap when considering all these aspects. In this work, we propose ReferDINO, a strong RVOS model that inherits region-level vision-language alignment from foundational visual grounding models, and is further endowed with pixel-level dense perception and cross-modal spatiotemporal reasoning. In detail, ReferDINO integrates two key components: 1) a grounding-guided deformable mask decoder that utilizes location prediction to progressively guide mask prediction through differentiable deformation mechanisms; 2) an object-consistent temporal enhancer that injects pretrained time-varying text features into inter-frame interaction to capture object-aware dynamic changes. Moreover, a confidence-aware query pruning strategy is designed to accelerate object decoding without compromising model performance. Extensive experimental results on five benchmarks demonstrate that our ReferDINO significantly outperforms previous methods (e.g., +3.9% (J&F) on Ref-YouTube-VOS) with real-time inference speed (51 FPS).