mlsanityData to 7 Oct 2026

Rethinking Cross-modal Interaction from a Top-down Perspective for Referring Video Object Segmentation

arXiv 2106.01061 ↗First posted 2 Jun 2021cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Referring video object segmentation (RVOS) aims to segment video objects with the guidance of natural language reference. Previous methods typically tackle RVOS through directly grounding linguistic reference over the image lattice. Such bottom-up strategy fails to explore object-level cues, easily leading to inferior results. In this work, we instead put forward a two-stage, top-down RVOS solution. First, an exhaustive set of object tracklets is constructed by propagating object masks detected from several sampled frames to the entire video. Second, a Transformer-based tracklet-language grounding module is proposed, which models instance-level visual relations and cross-modal interactions simultaneously and efficiently. Our model ranks first place on CVPR2021 Referring Youtube-VOS challenge.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

22 of 29 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2015Fast R-CNNcited by 2,930
  8. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  9. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  10. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  11. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  12. 2016Modeling Context in Referring Expressionscited by 1,054
10 more
  1. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  2. 2019Hybrid Task Cascade for Instance Segmentationcited by 479
  3. 2020ResNeSt: Split-Attention Networkscited by 445
  4. 2020Conditional Convolutions for Instance Segmentationcited by 314
  5. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  6. 2018Video Object Segmentation with Language Referring Expressionscited by 161
  7. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  8. 2018Actor and Action Video Segmentation from a Sentencecited by 105
  9. 2021Video Instance Segmentation with a Propose-Reduce Paradigmcited by 74
  10. 2021ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 18; 2024: 9; 2025: 6; 2026: 1.

42 papers.

Sort:

By area:

  1. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  2. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  3. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  4. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  5. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  6. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  7. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  8. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 99
  9. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  10. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  11. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  12. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
30 more
  1. 2022Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensuscited by 49
  2. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  3. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  4. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  5. 2023Tracking with Human-Intent Reasoningcited by 32
  6. 2023Temporal Collection and Distribution for Referring Video Object Segmentationcited by 27
  7. 2024Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentationcited by 27
  8. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  9. 2023LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentationcited by 23
  10. 2025Multimodal Referring Segmentation: A Surveycited by 22
  11. 2023Language-Guided 3D Object Detection in Point Cloud for Autonomous Drivingcited by 19
  12. 2024Temporally Consistent Referring Video Object Segmentation with Hybrid Memorycited by 14
  13. 2022SiRi: A Simple Selective Retraining Mechanism for Transformer-based Visual Groundingcited by 12
  14. 2022In-N-Out Generative Learning for Dense Unsupervised Video Segmentationcited by 7
  15. 2023Learning Referring Video Object Segmentation from Weak Annotationcited by 6
  16. 2024SafaRi:Adaptive Sequence Transformer for Weakly Supervised Referring Expression Segmentationcited by 6
  17. 2024Temporal-Enhanced Multimodal Transformer for Referring Multi-Object Tracking and Segmentationcited by 6
  18. 2025InterRVOS: Interaction-aware Referring Video Object Segmentationcited by 6
  19. 20221st Place Solution for YouTubeVOS Challenge 2022: Referring Video Object Segmentationcited by 5
  20. 2023Bidirectional Correlation-Driven Inter-Frame Interaction Transformer for Referring Video Object Segmentationcited by 5
  21. 2023Fully Transformer-Equipped Architecture for End-to-End Referring Video Object Segmentationcited by 4
  22. 2024UNINEXT-Cutie: The 1st Solution for LSVOS Challenge RVOS Trackcited by 3
  23. 2025The 1st Solution for 4th PVUW MeViS Challenge: Unleashing the Potential of Large Multimodal Models for Referring Video Segmentationcited by 3
  24. 20241st Place Solution for 5th LSVOS Challenge: Referring Video Object Segmentationcited by 2
  25. 2024Towards Flexible Visual Relationship Segmentationcited by 2
  26. 2024Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentationcited by 1
  27. 2023MarineVRS: Marine Video Retrieval System with Explainability via Semantic Understandingcited by 0
  28. 2025Mitigating Query Selection Bias in Referring Video Object Segmentationcited by 0
  29. 2025Referring Video Object Segmentation with Cross-Modality Proxy Queriescited by 0
  30. 2026Weakly-Supervised Referring Video Object Segmentation through Text Supervisioncited by 0