mlsanityData to 7 Oct 2026

XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model

arXiv 2207.07115 ↗First posted 14 Jul 2022cs.CVcited by 436

Not on the map. Why?

Abstract (arXiv)

We present XMem, a video object segmentation architecture for long videos with unified feature memory stores inspired by the Atkinson-Shiffrin memory model. Prior work on video object segmentation typically only uses one type of feature memory. For videos longer than a minute, a single feature memory model tightly links memory consumption and accuracy. In contrast, following the Atkinson-Shiffrin model, we develop an architecture that incorporates multiple independent yet deeply-connected feature memory stores: a rapidly updated sensory memory, a high-resolution working memory, and a compact thus sustained long-term memory. Crucially, we develop a memory potentiation algorithm that routinely consolidates actively used working memory elements into the long-term memory, which avoids memory explosion and minimizes performance decay for long-term prediction. Combined with a new memory reading mechanism, XMem greatly exceeds state-of-the-art performance on long-video datasets while being on par with state-of-the-art methods (that do not work on long videos) on short-video datasets. Code is available at https://hkchengrex.github.io/XMem

Key facts. Cited by 436 papers in the map (data through 7 Oct 2026). Builds on 51 of its 64 references that are in the map. First posted on arXiv 14 Jul 2022.

Builds on

51 of 64 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  7. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  8. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  9. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  10. 2016One-Shot Video Object Segmentationcited by 233
  11. 2021Associating Objects with Transformers for Video Object Segmentationcited by 225
  12. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
38 more
  1. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 200
  2. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  3. 2019FSS-1000: A 1000-Class Dataset for Few-Shot Segmentationcited by 160
  4. 2016Learning Video Object Segmentation from Static Imagescited by 151
  5. 2020Collaborative Video Object Segmentation by Foreground-Background Integrationcited by 142
  6. 2021Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusioncited by 129
  7. 2020Video Object Segmentation with Episodic Graph Memory Networkscited by 118
  8. 2020CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinementcited by 113
  9. 2020Kernelized Memory Network for Video Object Segmentationcited by 111
  10. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109
  11. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  12. 2020Video Object Segmentation with Adaptive Feature Bank and Uncertain-Region Refinementcited by 101
  13. 2017Video Object Segmentation Without Temporal Informationcited by 93
  14. 2019Towards High-Resolution Salient Object Detectioncited by 92
  15. 2020Learning What to Learn for Video Object Segmentationcited by 89
  16. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  17. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  18. 2020Collaborative Video Object Segmentation by Multi-Scale Foreground-Background Integrationcited by 81
  19. 2019BlenderProccited by 76
  20. 2020Learning Fast and Robust Target Models for Video Object Segmentationcited by 76
  21. 2018Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learningcited by 74
  22. 2021Hierarchical Memory Matching Network for Video Object Segmentationcited by 72
  23. 2021SwiftNet: Real-time Video Object Segmentationcited by 67
  24. 2018A Generative Appearance Model for End-to-end Video Object Segmentationcited by 66
  25. 2020Fast Video Object Segmentation using the Global Context Modulecited by 65
  26. 2021Learning Position and Target Consistency for Memory-based Video Object Segmentationcited by 63
  27. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  28. 2019RANet: Ranking Attention Network for Fast Video Object Segmentationcited by 60
  29. 2018Fast and Accurate Online Video Object Segmentation via Tracking Partscited by 57
  30. 2018Video Object Segmentation with Joint Re-identification and Attention-Aware Mask Propagationcited by 56
  31. 2020A Transductive Approach for Video Object Segmentationcited by 55
  32. 2020State-Aware Tracker for Real-Time Video Object Segmentationcited by 54
  33. 2018MaskRNN: Instance Level Video Object Segmentationcited by 48
  34. 2020Fast Video Object Segmentation With Temporal Aggregation Network and Dynamic Template Matchingcited by 37
  35. 2021Reliable Propagation-Correction Modulation for Video Object Segmentationcited by 36
  36. 2020Make One-Shot Video Object Segmentation Efficient Againcited by 30
  37. 2019CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routingcited by 29
  38. 2020Learning Dynamic Network Using a Reuse Gate Function in Semi-supervised Video Object Segmentationcited by 28

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 96; 2024: 132; 2025: 130; 2026: 72.

436 papers.

Sort:

By area:

  1. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  2. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  3. 2023Segment Everything Everywhere All at Oncecited by 546
  4. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  5. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  6. 2023Segment Anything in High Qualitycited by 373
  7. 2025Memory in the Age of AI Agentscited by 285
  8. 20243D Reconstruction with Spatial Memorycited by 269
  9. 2023Personalize Segment Anything Model with One Shotcited by 250
  10. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  11. 2023Track Anything: Segment Anything Meets Videoscited by 230
  12. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
38 more
  1. 2023SegGPT: Segmenting Everything In Contextcited by 193
  2. 2023Segment and Track Anythingcited by 193
  3. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  4. 2023Putting the Object Back into Video Object Segmentationcited by 182
  5. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  6. 2024Streaming Long Video Understanding with Large Language Modelscited by 164
  7. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  8. 2024Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streamscited by 160
  9. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  10. 2023Matcher: Segment Anything with One Shot Using All-Purpose Feature Matchingcited by 142
  11. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  12. 2024VISA: Reasoning Video Object Segmentation via Large Language Modelscited by 123
  13. 2024RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videoscited by 113
  14. 2024HourVideo: 1-Hour Video-Language Understandingcited by 110
  15. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  16. 2024PSALM: Pixelwise SegmentAtion with Large Multi-Modal Modelcited by 100
  17. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 99
  18. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  19. 2023Segment Anything Meets Point Trackingcited by 81
  20. 2023AVID: Any-Length Video Inpainting with Diffusion Modelcited by 81
  21. 2025Flash-VStream: Efficient Real-Time Understanding for Long Video Streamscited by 78
  22. 2024Local All-Pair Correspondence for Point Trackingcited by 77
  23. 2024SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Treecited by 77
  24. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  25. 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
  26. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  27. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  28. 2023Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Modelscited by 66
  29. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  30. 2024OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuningcited by 64
  31. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  32. 2024A Distractor-Aware Memory for Visual Object Tracking with SAM2cited by 63
  33. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 60
  34. 2023Language Models as Zero-Shot Trajectory Generatorscited by 59
  35. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  36. 2024Koala: Key frame-conditioned long video-LLMcited by 57
  37. 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
  38. 2023XMem++: Production-level Video Segmentation From Few Annotated Framescited by 53

50 most cited of 436 All 436 (JSON)