mlsanityData to 7 Oct 2026

SAM-MT: Real-Time Interactive Multi-Target Video Segmentation

arXiv 2607.08688 ↗First posted 9 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern Video Object Segmentation (VOS) involves tracking and segmenting user-specified targets. While recent approaches have achieved remarkable performance in single-target scenarios, extending them to multi-target settings typically involves replicating the single-target processing for each individual object, resulting in reduced frame rates (FPS) with unbounded latency as target count increases. Built upon Segment Anything 2 (SAM2), we propose SAM-MT, which addresses this by transforming the model into an interactive framework for real-time Multi-Target video segmentation. SAM-MT uses explicit queries to represent different individual targets, in parallel with a shared representation for global context. It employs decoupled masked attention to keep individual identities distinct from cross-target interference, and sparse memory for stable temporal evolution, along with specialized strategies for occlusion handling and overlap prevention. SAM-MT successfully decouples latency from the number of targets, achieving real-time speed on par with single-target baselines (>36 FPS for 10 targets) while maintaining SAM2's robust video segmentation performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 58 of its 60 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

58 of 60 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2023Segment Anythingcited by 8,158
  4. 2017Mask R-CNNcited by 7,259
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  6. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  7. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  10. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 770
  11. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  12. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
38 more
  1. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  2. 2023Segment Anything in High Qualitycited by 373
  3. 2022Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentationcited by 341
  4. 2022OneFormer: One Transformer to Rule Universal Image Segmentationcited by 327
  5. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  6. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  7. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  8. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  9. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  10. 2024A Survey on 3D Gaussian Splattingcited by 221
  11. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  12. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  13. 2023Putting the Object Back into Video Object Segmentationcited by 182
  14. 2021Mask2Former for Video Instance Segmentationcited by 169
  15. 2023Towards Open Vocabulary Learning: A Surveycited by 165
  16. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 154
  17. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 146
  18. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  19. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  20. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  21. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 99
  22. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 91
  23. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  24. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 77
  25. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  26. 2020Boundary-preserving Mask R-CNNcited by 74
  27. 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
  28. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  29. 2021Learning Position and Target Consistency for Memory-based Video Object Segmentationcited by 63
  30. 2024A Distractor-Aware Memory for Visual Object Tracking with SAM2cited by 63
  31. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  32. 2023XMem++: Production-level Video Segmentation From Few Annotated Framescited by 53
  33. 2022Recurrent Dynamic Embedding for Video Object Segmentationcited by 52
  34. 2024LVOS: A Benchmark for Large-scale Long-term Video Object Segmentationcited by 41
  35. 2022Per-Clip Video Object Segmentationcited by 40
  36. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  37. 2023FastInst: A Simple Query-Based Model for Real-Time Instance Segmentationcited by 32
  38. 2025Multimodal Referring Segmentation: A Surveycited by 22

50 most cited of 58 All 58 (JSON)

Built on by

Not cited yet.