mlsanityData to 14 Oct 2026

SAM 2: Segment Anything in Images and Videos

arXiv 2408.00714 ↗First posted 1 Aug 2024cs.CV cs.AI cs.LGcited by 2,863

Not on the map. Why?

Cited by 215 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present Segment Anything Model 2 (SAM 2), a foundation model towards solving promptable visual segmentation in images and videos. We build a data engine, which improves model and data via user interaction, to collect the largest video segmentation dataset to date. Our model is a simple transformer architecture with streaming memory for real-time video processing. SAM 2 trained on our data provides strong performance across a wide range of tasks. In video segmentation, we observe better accuracy, using 3x fewer interactions than prior approaches. In image segmentation, our model is more accurate and 6x faster than the Segment Anything Model (SAM). We believe that our data, model, and insights will serve as a significant milestone for video segmentation and related perception tasks. We are releasing our main model, dataset, as well as code for model training and our demo.

Key facts. Cited by 2,863 papers in the map (data through 14 Oct 2026). Builds on 92 of its 126 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

92 of 126 references.

References · 92

  1. 2023Segment Anythingcited by 8,234
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  7. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  8. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,217
  9. 2018Model Cards for Model Reportingcited by 1,080
  10. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  11. 2021Scaling Vision Transformerscited by 999
  12. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
38 more
  1. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  2. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  3. 2021Carbon Emissions and Large Neural Network Trainingcited by 567
  4. 2020Rescaling Egocentric Visioncited by 540
  5. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  6. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 518
  7. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 490
  8. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  9. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  10. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  11. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
  12. 2023Segment Anything in High Qualitycited by 376
  13. 2020Virtual KITTI 2cited by 353
  14. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  15. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  16. 2024Rotary Position Embedding for Vision Transformercited by 236
  17. 2016One-Shot Video Object Segmentationcited by 235
  18. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  19. 2023Track Anything: Segment Anything Meets Videoscited by 234
  20. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 232
  21. 2021Associating Objects with Transformers for Video Object Segmentationcited by 226
  22. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 203
  23. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  24. 2023Segment and Track Anythingcited by 193
  25. 2023Putting the Object Back into Video Object Segmentationcited by 188
  26. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 181
  27. 2021Reviving Iterative Training with Mask Guidance for Interactive Segmentationcited by 163
  28. 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
  29. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  30. 2016Learning Video Object Segmentation from Static Imagescited by 151
  31. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 148
  32. 2019WoodScape: A multi-task, multi-camera fisheye dataset for autonomous drivingcited by 144
  33. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  34. 2020Video Panoptic Segmentationcited by 142
  35. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 140
  36. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  37. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  38. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 254; 2025: 1305; 2026: 1304.

2,863 papers.

Sort:

By area:

  1. 2025VACE: All-in-One Video Creation and Editingcited by 479
  2. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  3. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
  4. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  5. 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 296
  6. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  7. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  8. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
  9. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  10. 2025Video models are zero-shot learners and reasonerscited by 220
  11. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modelingcited by 211
  12. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
38 more
  1. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  2. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  3. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  4. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 158
  5. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  6. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
  7. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
  8. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  9. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
  10. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 102
  11. 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 96
  12. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  13. 2025PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyondcited by 95
  14. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 94
  15. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  16. 2024DINO-X: A Unified Vision Model for Open-World Object Detection and Understandingcited by 89
  17. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  18. 2025JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronizationcited by 88
  19. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
  20. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  21. 2025Visual Planning: Let's Think Only with Imagescited by 85
  22. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 84
  23. 2025Describe Anything: Detailed Localized Image and Video Captioningcited by 84
  24. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
  25. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 82
  26. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  27. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 80
  28. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 80
  29. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  30. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 75
  31. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 74
  32. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  33. 2025Detect Anything via Next Point Predictioncited by 73
  34. 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 72
  35. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
  36. 2026SAM 3D Body: Robust Full-Body Human Mesh Recoverycited by 69
  37. 2025SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoningcited by 68
  38. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 68

50 most cited of 2,863 All 2,863 (JSON)