mlsanityData to 7 Oct 2026

Video Object of Interest Segmentation

arXiv 2212.02871 ↗First posted 6 Dec 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

In this work, we present a new computer vision task named video object of interest segmentation (VOIS). Given a video and a target image of interest, our objective is to simultaneously segment and track all objects in the video that are relevant to the target image. This problem combines the traditional video object segmentation task with an additional image indicating the content that users are concerned with. Since no existing dataset is perfectly suitable for this new task, we specifically construct a large-scale dataset called LiveVideos, which contains 2418 pairs of target images and live videos with instance-level annotations. In addition, we propose a transformer-based method for this task. We revisit Swin Transformer and design a dual-path structure to fuse video and image features. Then, a transformer decoder is employed to generate object proposals for segmentation and tracking from the fused features. Extensive experiments on LiveVideos dataset show the superiority of our proposed method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 58 references that are in the map. First posted on arXiv 6 Dec 2022.

Builds on

41 of 58 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2021Video Swin Transformercited by 1,090
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  11. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  12. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
29 more
  1. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  2. 2019Video Instance Segmentationcited by 391
  3. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  4. 2016One-Shot Video Object Segmentationcited by 233
  5. 2019MOTS: Multi-Object Tracking and Segmentationcited by 222
  6. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  7. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  8. 2016Learning Video Object Segmentation from Static Imagescited by 151
  9. 2017Detect to Track and Track to Detectcited by 131
  10. 2019Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagationcited by 125
  11. 2020STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videoscited by 121
  12. 2020Video Object Segmentation with Episodic Graph Memory Networkscited by 118
  13. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  14. 2020Learning What to Learn for Video Object Segmentationcited by 89
  15. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  16. 2021SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentationcited by 87
  17. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  18. 2020Zero-Shot Video Object Segmentation via Attentive Graph Neural Networkscited by 84
  19. 2017Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependenciescited by 76
  20. 2021Video Instance Segmentation with a Propose-Reduce Paradigmcited by 74
  21. 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
  22. 2021Learning Position and Target Consistency for Memory-based Video Object Segmentationcited by 63
  23. 2020Motion-Attentive Transition for Zero-Shot Video Object Segmentationcited by 62
  24. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  25. 2020Learning Video Object Segmentation from Unlabeled Videoscited by 42
  26. 2021Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentationcited by 42
  27. 2020UnOVOST: Unsupervised Offline Video Object Segmentation and Trackingcited by 34
  28. 2020Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentationcited by 28
  29. 2021Target-Aware Object Discovery and Association for Unsupervised Video Multi-Object Segmentationcited by 21

Built on by

Not cited yet.