mlsanityData to 14 Oct 2026

Putting the Object Back into Video Object Segmentation

arXiv 2310.12982 ↗First posted 19 Oct 2023cs.CVcited by 188

Not on the map. Why?

Abstract (arXiv)

We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result. Recent works on VOS employ bottom-up pixel-level memory reading which struggles due to matching noise, especially in the presence of distractors, resulting in lower performance in more challenging data. In contrast, Cutie performs top-down object-level memory reading by adapting a small set of object queries. Via those, it interacts with the bottom-up pixel features iteratively with a query-based object transformer (qt, hence Cutie). The object queries act as a high-level summary of the target object, while high-resolution feature maps are retained for accurate segmentation. Together with foreground-background masked attention, Cutie cleanly separates the semantics of the foreground object from the background. On the challenging MOSE dataset, Cutie improves by 8.7 J&F over XMem with a similar running time and improves by 4.2 J&F over DeAOT while being three times faster. Code is available at: https://hkchengrex.github.io/Cutie

Key facts. Cited by 188 papers in the map (data through 14 Oct 2026). Builds on 78 of its 97 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

78 of 97 references.

References · 78

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2023Segment Anythingcited by 8,234
  6. 2020End-to-End Object Detection with Transformerscited by 7,414
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  8. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  10. 2020Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentationcited by 596
  11. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 575
  12. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 552
38 more
  1. 2019PointRend: Image Segmentation as Renderingcited by 440
  2. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  3. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  4. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
  5. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  6. 2016One-Shot Video Object Segmentationcited by 235
  7. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 234
  8. 2023Track Anything: Segment Anything Meets Videoscited by 234
  9. 2021Associating Objects with Transformers for Video Object Segmentationcited by 226
  10. 2023Segment and Track Anythingcited by 193
  11. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  12. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  13. 2021Reviving Iterative Training with Mask Guidance for Interactive Segmentationcited by 163
  14. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  15. 2021Occluded Video Instance Segmentation: A Benchmarkcited by 154
  16. 2016Learning Video Object Segmentation from Static Imagescited by 151
  17. 2022Decoupling Features in Hierarchical Propagation for Video Object Segmentationcited by 148
  18. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  19. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  20. 2020Video Object Segmentation with Episodic Graph Memory Networkscited by 118
  21. 2020CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinementcited by 113
  22. 2020Kernelized Memory Network for Video Object Segmentationcited by 111
  23. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109
  24. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  25. 2020Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinementcited by 101
  26. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  27. 2017Video Object Segmentation Without Temporal Informationcited by 93
  28. 2022Towards Grand Unification of Object Trackingcited by 93
  29. 2019Towards High-Resolution Salient Object Detectioncited by 92
  30. 2018VideoMatch: Matching based Video Object Segmentationcited by 90
  31. 2020Learning What to Learn for Video Object Segmentationcited by 89
  32. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  33. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  34. 2020Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integrationcited by 81
  35. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 77
  36. 2020Learning Fast and Robust Target Models for Video Object Segmentationcited by 76
  37. 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
  38. 2021SwiftNet: Real-time Video Object Segmentationcited by 67

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 48; 2025: 80; 2026: 57.

188 papers.

Sort:

By area:

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
  3. 20243D Reconstruction with Spatial Memorycited by 277
  4. 2025Sa2VA: Marrying SAM2 with MLLM for Dense Grounded Understanding of Images and Videoscited by 159
  5. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 104
  6. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  7. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  8. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 80
  9. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
  10. 2024A Distractor-Aware Memory for Visual Object Tracking with SAM2cited by 66
  11. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 62
  12. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 50
38 more
  1. 2024Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruningcited by 47
  2. 2024Segment anything model 2: an application to 2D and 3D medical imagescited by 28
  3. 2024UniVS: Unified and Universal Video Segmentation with Prompts as Queriescited by 27
  4. 2025Dynamic Camera Poses and Where to Find Themcited by 27
  5. 2025Advancing Complex Video Object Segmentation via Progressive Concept Constructioncited by 26
  6. 2025MatAnyone: Stable Video Matting with Consistent Memory Propagationcited by 24
  7. 2023UniRef++: Segment Every Reference Object in Spatial and Temporal Spacescited by 22
  8. 2024RMem: Restricted Memory Banks Improve Video Object Segmentationcited by 22
  9. 2024Efficient Track Anythingcited by 21
  10. 2025EdgeTAM: On-Device Track Anything Modelcited by 21
  11. 2025Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrationscited by 19
  12. 2024ViCaS: A Dataset for Combining Holistic and Pixel-level Video Understanding using Captions with Grounded Segmentationcited by 17
  13. 2025MagicHOI: Leveraging 3D Priors for Accurate Hand-object Reconstruction from Short Monocular Video Clipscited by 17
  14. 2024Language-Guided Object-Centric Diffusion Policy for Generalizable and Collision-Aware Robotic Manipulationcited by 14
  15. 2024T-3DGS: Removing Transient Objects for 3D Scene Reconstructioncited by 13
  16. 2025SAM2 for Image and Video Segmentation: A Comprehensive Surveycited by 13
  17. 2024PVUW 2024 Challenge on Complex Video Understanding: Methods and Resultscited by 12
  18. 2024Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagationcited by 12
  19. 2024IKEA Manuals at Work: 4D Grounding of Assembly Instructions on Internet Videoscited by 12
  20. 2024D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Videocited by 11
  21. 2025Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learningcited by 10
  22. 2025Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Groundingcited by 10
  23. 2024LVMark: Robust Watermark for Latent Video Diffusion Modelscited by 9
  24. 2025CoT-RVS: Zero-Shot Chain-of-Thought Reasoning Segmentation for Videoscited by 9
  25. 2024LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentationcited by 8
  26. 2024Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Modelscited by 8
  27. 2025From Simple to Complex Skills: The Case of In-Hand Object Reorientationcited by 8
  28. 2025SAM-I2V: Upgrading SAM to Support Promptable Video Segmentation with Less than 0.2% Training Costcited by 8
  29. 2025SAM-Body4D: Training-Free 4D Human Body Mesh Recovery from Videoscited by 8
  30. 2024Learning Spatial-Semantic Features for Robust Video Object Segmentationcited by 7
  31. 2024LiVOS: Light Video Object Segmentation with Gated Linear Matchingcited by 7
  32. 2025MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentationcited by 7
  33. 2025One Trajectory, One Token: Grounded Video Tokenization via Panoptic Sub-object Trajectorycited by 7
  34. 2025Unified Open-World Segmentation with Multi-Modal Promptscited by 7
  35. 2024R2-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbationscited by 6
  36. 2024CellSeg1: Robust Cell Segmentation with One Training Imagecited by 6
  37. 2025No Train Yet Gain: Towards Generic Multi-Object Tracking in Sports and Beyondcited by 6
  38. 2025HiM2SAM: Enhancing SAM2 with Hierarchical Motion Estimation and Memory Optimization towards Long-term Trackingcited by 6

50 most cited of 188 All 188 (JSON)