mlsanityData to 7 Oct 2026

Segment and Track Anything

arXiv 2305.06558 ↗First posted 11 May 2023cs.CVcited by 193

Not on the map. Why?

Abstract (arXiv)

This report presents a framework called Segment And Track Anything (SAMTrack) that allows users to precisely and effectively segment and track any object in a video. Additionally, SAM-Track employs multimodal interaction methods that enable users to select multiple objects in videos for tracking, corresponding to their specific requirements. These interaction methods comprise click, stroke, and text, each possessing unique benefits and capable of being employed in combination. As a result, SAM-Track can be used across an array of fields, ranging from drone technology, autonomous driving, medical imaging, augmented reality, to biological analysis. SAM-Track amalgamates Segment Anything Model (SAM), an interactive key-frame segmentation model, with our proposed AOT-based tracking model (DeAOT), which secured 1st place in four tracks of the VOT 2022 challenge, to facilitate object tracking in video. In addition, SAM-Track incorporates Grounding-DINO, which enables the framework to support text-based interaction. We have demonstrated the remarkable capabilities of SAM-Track on DAVIS-2016 Val (92.0%), DAVIS-2017 Test (79.2%)and its practicability in diverse applications. The project page is available at: https://github.com/z-x-yang/Segment-and-Track-Anything.

Key facts. Cited by 193 papers in the map (data through 7 Oct 2026). Builds on 17 of its 23 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

17 of 23 references.

References · 17

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  4. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  5. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  6. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  7. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 146
  8. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  9. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  10. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  11. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  12. 2020Video Object Segmentation with Episodic Graph Memory Networkscited by 118
5 more
  1. 2020Zero-Shot Video Object Segmentation via Attentive Graph Neural Networkscited by 84
  2. 2020Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integrationcited by 81
  3. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  4. 2022Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentationcited by 35
  5. 2023Video Object Segmentation in Panoptic Wild Scenescited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 44; 2024: 95; 2025: 41; 2026: 13.

193 papers.

Sort:

By area:

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  2. 2023LangSplat: 3D Language Gaussian Splattingcited by 410
  3. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  4. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  5. 2023EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anythingcited by 187
  6. 2023Putting the Object Back into Video Object Segmentationcited by 182
  7. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  8. 2024L4GM: Large 4D Gaussian Reconstruction Modelcited by 143
  9. 2024OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understandingcited by 126
  10. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  11. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  12. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
38 more
  1. 2023Segment Anything Meets Point Trackingcited by 81
  2. 2023Efficient Emotional Adaptation for Audio-Driven Talking-Head Generationcited by 81
  3. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  4. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 76
  5. 2024Open-Vocabulary SAM: Segment and Recognize Twenty-thousand Classes Interactivelycited by 74
  6. 2024OMG-Seg: Is One Model Good Enough For All Segmentation?cited by 73
  7. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  8. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  9. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  10. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  11. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Priorcited by 55
  12. 2023CATR: Combinatorial-Dependence Audio-Queried Transformer for Audio-Visual Video Segmentationcited by 52
  13. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  14. 2024A Survey for Foundation Models in Autonomous Drivingcited by 51
  15. 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
  16. 2024Matching Anything by Segmenting Anythingcited by 49
  17. 2025Perceive Anything: Recognize, Explain, Caption, and Segment Anything in Images and Videoscited by 49
  18. 2023A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineeringcited by 45
  19. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  20. 2024Gaga: Group Any Gaussians via 3D-aware Memory Bankcited by 40
  21. 2023Matte Anything: Interactive Natural Image Matting with Segment Anything Modelscited by 39
  22. 2024RobustSAM: Segment Anything Robustly on Degraded Imagescited by 36
  23. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  24. 2023TinySAM: Pushing the Envelope for Efficient Segment Anything Modelcited by 33
  25. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  26. 2023Robustness of SAM: Segment Anything Under Corruptions and Beyondcited by 30
  27. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  28. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  29. 2023Follow Anything: Open-set detection, tracking, and following in real-timecited by 28
  30. 2024Foundation Models for Biomedical Image Segmentation: A Surveycited by 28
  31. 2024GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentationcited by 27
  32. 2024GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splattingcited by 25
  33. 2023RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentationcited by 24
  34. 2024ViLLa: Video Reasoning Segmentation with Large Language Modelcited by 23
  35. 2024OV-NeRF: Open-vocabulary Neural Radiance Fields with Vision and Language Foundation Models for 3D Semantic Understandingcited by 22
  36. 2024Object Segmentation-Assisted Inter Prediction for Versatile Video Codingcited by 22
  37. 2023MagDiff: Multi-Alignment Diffusion for High-Fidelity Video Generation and Editingcited by 21
  38. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 21

50 most cited of 193 All 193 (JSON)