mlsanityData to 7 Oct 2026

Flow-guided Semi-supervised Video Object Segmentation

arXiv 2301.10492 ↗First posted 25 Jan 2023cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose an optical flow-guided approach for semi-supervised video object segmentation. Optical flow is usually exploited as additional guidance information in unsupervised video object segmentation. However, its relevance in semi-supervised video object segmentation has not been fully explored. In this work, we follow an encoder-decoder approach to address the segmentation task. A model to extract the combined information from optical flow and the image is proposed, which is then used as input to the target model and the decoder network. Unlike previous methods where concatenation is used to integrate information from image data and optical flow, a simple yet effective attention mechanism is exploited in our work. Experiments on DAVIS 2017 and YouTube-VOS 2019 show that by integrating the information extracted from optical flow into the original image branch results in a strong performance gain and our method achieves state-of-the-art performance.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 50 of its 59 references that are in the map. First posted on arXiv 25 Jan 2023.

Builds on

50 of 59 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Mask R-CNNcited by 7,259
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  10. 2017Non-local Neural Networkscited by 2,290
  11. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  12. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
38 more
  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  4. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  5. 2020Pre-Trained Image Processing Transformercited by 891
  6. 2018Image Transformercited by 770
  7. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  8. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  9. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  10. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  11. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  12. 2019Attention Augmented Convolutional Networkscited by 363
  13. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 346
  14. 2018Video Action Transformer Networkcited by 316
  15. 2020Exploring Self-attention for Image Recognitioncited by 313
  16. 2019Local Relation Networks for Image Recognitioncited by 210
  17. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  18. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  19. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  20. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  21. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
  22. 2018Learning a Discriminative Feature Network for Semantic Segmentationcited by 114
  23. 2020Kernelized Memory Network for Video Object Segmentationcited by 111
  24. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109
  25. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  26. 2020Learning What to Learn for Video Object Segmentationcited by 89
  27. 2018PReMVOS: Proposal-generation, Refinement and Merging for Video Object Segmentationcited by 81
  28. 2020Learning Fast and Robust Target Models for Video Object Segmentationcited by 76
  29. 2020Spatial Temporal Transformer Network for Skeleton-based Action Recognitioncited by 76
  30. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  31. 2018A Generative Appearance Model for End-to-end Video Object Segmentationcited by 66
  32. 2020Fast Video Object Segmentation using the Global Context Modulecited by 65
  33. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  34. 2016Learning Motion Patterns in Videoscited by 53
  35. 2017Learning to Segment Moving Objectscited by 39
  36. 2021Video Class Agnostic Segmentation Benchmark for Autonomous Drivingcited by 21
  37. 2016Deep Motion Features for Visual Trackingcited by 4
  38. 2021Normalized Convolution Upsampling for Refined Optical Flow Estimationcited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Mask Propagation for Efficient Video Semantic Segmentationcited by 26