mlsanityData to 7 Oct 2026

SMITE: Segment Me In TimE

arXiv 2410.18538 ↗First posted 24 Oct 2024cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Segmenting an object in a video presents significant challenges. Each pixel must be accurately labelled, and these labels must remain consistent across frames. The difficulty increases when the segmentation is with arbitrary granularity, meaning the number of segments can vary arbitrarily, and masks are defined based on only one or a few sample images. In this paper, we address this issue by employing a pre-trained text to image diffusion model supplemented with an additional tracking mechanism. We demonstrate that our approach can effectively manage various segmentation scenarios and outperforms state-of-the-art alternatives.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 79 of its 89 references that are in the map. First posted on arXiv 24 Oct 2024.

Builds on

79 of 89 references.

References · 79

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2023Segment Anythingcited by 8,158
  8. 2017Mask R-CNNcited by 7,259
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,276
38 more
  1. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  2. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  3. 2016Pyramid Scene Parsing Networkcited by 2,472
  4. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  7. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  8. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  9. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  10. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  11. 2022Language-driven Semantic Segmentationcited by 711
  12. 2023CoTracker: It is Better to Track Togethercited by 516
  13. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  14. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  15. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  16. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  17. 2019Video Instance Segmentationcited by 391
  18. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  19. 2022MAGVIT: Masked Generative Video Transformercited by 341
  20. 2023VideoFusion: Decomposed Diffusion Models for High-Quality Video Generationcited by 330
  21. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 324
  22. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  23. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  24. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  25. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  26. 2014Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Partscited by 230
  27. 2018YouTube-VOS: Sequence-to-Sequence Video Object Segmentationcited by 228
  28. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  29. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  30. 2023FreeU: Free Lunch in Diffusion U-Netcited by 195
  31. 2023SegGPT: Segmenting Everything In Contextcited by 193
  32. 2016Deep Feature Flow for Video Recognitioncited by 186
  33. 2023FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editingcited by 165
  34. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  35. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  36. 2020Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinementcited by 101
  37. 2023FreeInit: Bridging Initialization Gap in Video Diffusion Modelscited by 93
  38. 2021Repurposing GANs for One-shot Semantic Part Segmentationcited by 88

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 7.

7 papers.

  1. 2025Articulate That Object Part (ATOP): 3D Part Articulation via Text and Motion Personalizationcited by 7
  2. 2025ASIA: Adaptive 3D Segmentation using Few Image Annotationscited by 6
  3. 2025Studying Image Diffusion Features for Zero-Shot Video Object Segmentationcited by 4
  4. 2025CountLoop: Training-Free High-Instance Image Generation via Iterative Agent Guidancecited by 4
  5. 2025Diffusion-Based Action Recognition Generalizes to Untrained Domainscited by 2
  6. 2025SAM2Auto: Auto Annotation Using FLASHcited by 0
  7. 2025Image Diffusion Models Exhibit Emergent Temporal Propagation in Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Explore In-Context Segmentation via Latent Diffusion Models
  2. 2022A Generalist Framework for Panoptic Segmentation of Images and Videos
  3. 2024Using Diffusion Priors for Video Amodal Segmentation
  4. 2024Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentation
  5. 2025Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation
  6. 2023SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process
  7. 2023Open-vocabulary Object Segmentation with Diffusion Models
  8. 2025SPOC: Spatially-Progressing Object State Change Segmentation in Video