mlsanityData to 14 Oct 2026

Shift and matching queries for video semantic segmentation

arXiv 2410.07635 ↗First posted 10 Oct 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Video segmentation is a popular task, but applying image segmentation models frame-by-frame to videos does not preserve temporal consistency. In this paper, we propose a method to extend a query-based image segmentation model to video using feature shift and query matching. The method uses a query-based architecture, where decoded queries represent segmentation masks. These queries should be matched before performing the feature shift to ensure that the shifted queries represent the same mask across different frames. Experimental results on CityScapes-VPS and VSPW show significant improvements from the baselines, highlighting the method's effectiveness in enhancing segmentation quality while efficiently reusing pre-trained weights.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 32 references that are in the map. First posted on arXiv 10 Oct 2024.

Builds on

25 of 32 references.

References · 25

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2017Mask R-CNNcited by 7,277
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  8. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  9. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  10. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  11. 2018Panoptic Segmentationcited by 802
  12. 2020Image Segmentation Using Deep Learning: A Surveycited by 660
13 more
  1. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 344
  2. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 328
  3. 2022Frozen CLIP Models are Efficient Video Learnerscited by 182
  4. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 179
  5. 2020Video Panoptic Segmentationcited by 142
  6. 2018Accel: A Corrective Fusion Network for Efficient Semantic Segmentation on Videocited by 78
  7. 2020Temporally Distributed Networks for Fast Video Semantic Segmentationcited by 77
  8. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  9. 2021Token Shift Transformer for Video Classificationcited by 61
  10. 2023CTVIS: Consistent Training for Online Video Instance Segmentationcited by 51
  11. 2023TarViS: A Unified Approach for Target-based Video Segmentationcited by 33
  12. 2023Tube-Link: A Flexible Cross Tube Framework for Universal Video Segmentationcited by 23
  13. 2022Vision Transformer with Cross-attention by Temporal Shift for Efficient Action Recognitioncited by 5

Built on by

Not cited yet.