mlsanityData to 14 Oct 2026

Video Object Segmentation with Language Referring Expressions

arXiv 1803.08006 ↗First posted 21 Mar 2018cs.CVcited by 162

Not on the map. Why?

Abstract (arXiv)

Most state-of-the-art semi-supervised video object segmentation methods rely on a pixel-accurate mask of a target object provided for the first frame of a video. However, obtaining a detailed segmentation mask is expensive and time-consuming. In this work we explore an alternative way of identifying a target object, namely by employing language referring expressions. Besides being a more practical and natural way of pointing out a target object, using language specifications can help to avoid drift as well as make the system more robust to complex dynamics and appearance variations. Leveraging recent advances of language grounding models designed for images, we propose an approach to extend them to video data, ensuring temporally coherent predictions. To evaluate our method we augment the popular video object segmentation benchmarks, DAVIS'16 and DAVIS'17 with language descriptions of target objects. We show that our language-supervised approach performs on par with the methods which have access to a pixel-level mask of the target object on DAVIS'16 and is competitive to methods using scribbles on the challenging DAVIS'17 dataset.

Key facts. Cited by 162 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 17; 2023: 30; 2024: 36; 2025: 46; 2026: 26.

162 papers.

Sort:

By area:

  1. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 228
  2. 2022VRT: A Video Restoration Transformercited by 220
  3. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  4. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  5. 2022Language as Queries for Referring Video Object Segmentationcited by 182
  6. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 170
  7. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  8. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 148
  9. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  10. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 127
  11. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 111
  12. 2023Language Prompt for Autonomous Drivingcited by 111
38 more
  1. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
  2. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 100
  3. 2023Referring Multi-Object Trackingcited by 93
  4. 2023Audio-Visual Segmentation with Semanticscited by 75
  5. 2023CLIP as RNN: Segment Countless Visual Concepts without Training Endeavorcited by 69
  6. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  7. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  8. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  9. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 56
  10. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  11. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  12. 2024Toward Visual Grounding: A Surveycited by 52
  13. 2024VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videoscited by 51
  14. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  15. 2022Towards Robust Referring Image Segmentationcited by 49
  16. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  17. 2024Elysium: Exploring Object-level Perception in Videos via MLLMcited by 48
  18. 2024SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentationcited by 44
  19. 2021Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentationcited by 42
  20. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  21. 2022Hierarchical Feature Alignment Network for Unsupervised Video Object Segmentationcited by 37
  22. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 35
  23. 2023Tracking with Human-Intent Reasoningcited by 33
  24. 2024Ref-AVS: Refer and Segment Objects in Audio-Visual Scenescited by 33
  25. 2025ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundationscited by 30
  26. 2021Embracing Uncertainty: Decoupling and De-bias for Robust Temporal Groundingcited by 29
  27. 2021Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Networkcited by 28
  28. 2023Temporal Collection and Distribution for Referring Video Object Segmentationcited by 27
  29. 2024Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentationcited by 27
  30. 2021Segmentation-grounded Scene Graph Generationcited by 25
  31. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 25
  32. 2021Revisiting Temporal Alignment for Video Restorationcited by 24
  33. 2023Connecting Vision and Language with Video Localized Narrativescited by 24
  34. 2023Type-to-Track: Retrieve Any Object via Prompt-based Trackingcited by 24
  35. 2025Back on Track: Bundle Adjustment for Dynamic Scene Reconstructioncited by 24
  36. 2023LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentationcited by 23
  37. 2023UniRef++: Segment Every Reference Object in Spatial and Temporal Spacescited by 22
  38. 2024Bootstrapping Referring Multi-Object Trackingcited by 22

50 most cited of 162 All 162 (JSON)