mlsanityData to 7 Oct 2026

Multi-Context Temporal Consistent Modeling for Referring Video Object Segmentation

arXiv 2501.04939 ↗First posted 9 Jan 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Referring video object segmentation aims to segment objects within a video corresponding to a given text description. Existing transformer-based temporal modeling approaches face challenges related to query inconsistency and the limited consideration of context. Query inconsistency produces unstable masks of different objects in the middle of the video. The limited consideration of context leads to the segmentation of incorrect objects by failing to adequately account for the relationship between the given text and instances. To address these issues, we propose the Multi-context Temporal Consistency Module (MTCM), which consists of an Aligner and a Multi-Context Enhancer (MCE). The Aligner removes noise from queries and aligns them to achieve query consistency. The MCE predicts text-relevant queries by considering multi-context. We applied MTCM to four different models, increasing performance across all of them, particularly achieving 47.6 J&F on the MeViS. Code is available at https://github.com/Choi58/MTCM.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 22 of its 51 references that are in the map. First posted on arXiv 9 Jan 2025.

Builds on

22 of 51 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  5. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  6. 2021Video Swin Transformercited by 1,090
  7. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  8. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  9. 2018Video Object Segmentation with Language Referring Expressionscited by 161
  10. 2021End-to-End Referring Video Object Segmentation with Multimodal Transformerscited by 135
  11. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  12. 2018Actor and Action Video Segmentation from a Sentencecited by 105
10 more
  1. 2023DVIS: Decoupled Video Instance Segmentation Frameworkcited by 77
  2. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  3. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  4. 2022Language-Bridged Spatial-Temporal Interaction for Referring Video Object Segmentationcited by 50
  5. 2023Spectrum-guided Multi-granularity Referring Video Object Segmentationcited by 48
  6. 2021Collaborative Spatial-Temporal Modeling for Language-Queried Video Actor Segmentationcited by 32
  7. 2021Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Networkcited by 28
  8. 2023Temporal Collection and Distribution for Referring Video Object Segmentationcited by 27
  9. 2019Discretely-constrained deep network for weakly supervised segmentationcited by 12
  10. 2023WAVER: Writing-style Agnostic Text-Video Retrieval via Distilling Vision-Language Models Through Open-Vocabulary Knowledgecited by 4

Built on by

Not cited yet.