# RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation

arXiv: https://arxiv.org/abs/2010.00263 · HTML: https://mlsanity.com/paper/2010.00263/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 23 of 23)

- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Language as Queries for Referring Video Object Segmentation](https://mlsanity.com/paper/2201.00487/) (2022)
- [Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding](https://mlsanity.com/paper/2601.10611/) (2026)
- [End-to-End Referring Video Object Segmentation with Multimodal Transformers](https://mlsanity.com/paper/2111.14821/) (2021)
- [Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation](https://mlsanity.com/paper/2305.16318/) (2023)
- [Towards Robust Referring Image Segmentation](https://mlsanity.com/paper/2209.09554/) (2022)
- [RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation](https://mlsanity.com/paper/2307.00997/) (2023)
- [Modeling Motion with Multi-Modal Features for Text-Based Video Segmentation](https://mlsanity.com/paper/2204.02547/) (2022)
- [Video Question Answering with Iterative Video-Text Co-Tokenization](https://mlsanity.com/paper/2208.00934/) (2022)
- [Visual Semantic Segmentation Based on Few/Zero-Shot Learning: An Overview](https://mlsanity.com/paper/2211.08352/) (2022)
- [Dense Video Object Captioning from Disjoint Supervision](https://mlsanity.com/paper/2306.11729/) (2023)
- [ReferEverything: Towards Segmenting Everything We Can Speak of in Videos](https://mlsanity.com/paper/2410.23287/) (2024)
- [Adversarial Attacks on Video Object Segmentation with Hard Region Discovery](https://mlsanity.com/paper/2309.13857/) (2023)
- [MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentation](https://mlsanity.com/paper/2501.13667/) (2025)
- [1st Place Solution for YouTubeVOS Challenge 2022: Referring Video Object Segmentation](https://mlsanity.com/paper/2212.14679/) (2022)
- [Bidirectional Correlation-Driven Inter-Frame Interaction Transformer for Referring Video Object Segmentation](https://mlsanity.com/paper/2307.00536/) (2023)
- [Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentation](https://mlsanity.com/paper/2309.11933/) (2023)
- [1st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentation](https://mlsanity.com/paper/2401.00663/) (2024)
- [Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems](https://mlsanity.com/paper/2506.14096/) (2025)
- [SynthRef: Generation of Synthetic Referring Expressions for Object Segmentation](https://mlsanity.com/paper/2106.04403/) (2021)
- [ReferDINO-Plus: 2nd Solution for 4th PVUW MeViS Challenge at CVPR 2025](https://mlsanity.com/paper/2503.23509/) (2025)
- [Referring Video Object Segmentation with Cross-Modality Proxy Queries](https://mlsanity.com/paper/2511.21139/) (2025)
- [Event-Driven Refresh and Recurrence Memory to Reduce Stale Grounding in Referring Video Object Segmentation](https://mlsanity.com/paper/2609.38758/) (2026)

## Abstract (arXiv)

The task of video object segmentation with referring expressions (language-guided VOS) is to, given a linguistic phrase and a video, generate binary masks for the object to which the phrase refers. Our work argues that existing benchmarks used for this task are mainly composed of trivial cases, in which referents can be identified with simple phrases. Our analysis relies on a new categorization of the phrases in the DAVIS-2017 and Actor-Action datasets into trivial and non-trivial REs, with the non-trivial REs annotated with seven RE semantic categories. We leverage this data to analyze the results of RefVOS, a novel neural network that obtains competitive results for the task of language-guided image segmentation and state of the art results for language-guided VOS. Our study indicates that the major challenges for the task are related to understanding motion and static actions.