mlsanityData to 7 Oct 2026

Video Object Segmentation using Space-Time Memory Networks

arXiv 1904.00607 ↗First posted 1 Apr 2019cs.CVcited by 436

Not on the map. Why?

Abstract (arXiv)

We propose a novel solution for semi-supervised video object segmentation. By the nature of the problem, available cues (e.g. video frame(s) with object masks) become richer with the intermediate predictions. However, the existing methods are unable to fully exploit this rich source of information. We resolve the issue by leveraging memory networks and learn to read relevant information from all available sources. In our framework, the past frames with object masks form an external memory, and the current frame as the query is segmented using the mask information in the memory. Specifically, the query and the memory are densely matched in the feature space, covering all the space-time pixel locations in a feed-forward fashion. Contrast to the previous approaches, the abundant use of the guidance information allows us to better handle the challenges such as appearance changes and occlussions. We validate our method on the latest benchmark sets and achieved the state-of-the-art performance (overall score of 79.4 on Youtube-VOS val set, J of 88.7 and 79.2 on DAVIS 2016/2017 val set respectively) while having a fast runtime (0.16 second/frame on DAVIS 2016 val set).

Key facts. Cited by 436 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 82; 2022: 87; 2023: 88; 2024: 75; 2025: 61; 2026: 43.

436 papers.

Sort:

By area:

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  4. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  5. 2021Learning Spatio-Temporal Transformer for Visual Trackingcited by 348
  6. 2022MixFormer: End-to-End Tracking with Iterative Mixed Attentioncited by 278
  7. 20243D Reconstruction with Spatial Memorycited by 269
  8. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  9. 2023Track Anything: Segment Anything Meets Videoscited by 230
  10. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  11. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  12. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
38 more
  1. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  2. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  3. 2023SegGPT: Segmenting Everything In Contextcited by 193
  4. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  5. 2023Putting the Object Back into Video Object Segmentationcited by 182
  6. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  7. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  8. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  9. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  10. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 146
  11. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  12. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  13. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  14. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  15. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 123
  16. 2022MeMOT: Multi-Object Tracking with Memorycited by 113
  17. 2024HourVideo: 1-Hour Video-Language Understandingcited by 110
  18. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  19. 2022Towards Grand Unification of Object Trackingcited by 93
  20. 2021STMTrack: Template-free Visual Tracking with Space-time Memory Networkscited by 91
  21. 2021Long Short-Term Transformer for Online Action Detectioncited by 87
  22. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  23. 2021Crossover Learning for Fast Online Video Instance Segmentationcited by 83
  24. 2023Segment Anything Meets Point Trackingcited by 81
  25. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  26. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 77
  27. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  28. 2021Video Instance Segmentation with a Propose-Reduce Paradigmcited by 74
  29. 2021Relational Embedding for Few-Shot Classificationcited by 74
  30. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 73
  31. 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
  32. 2023HIPTrack: Visual Tracking with Historical Promptscited by 71
  33. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  34. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  35. 2022BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Videocited by 66
  36. 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 64
  37. 2021Learning Position and Target Consistency for Memory-based Video Object Segmentationcited by 63
  38. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63

50 most cited of 436 All 436 (JSON)