mlsanityData to 7 Oct 2026

Appearance-Based Refinement for Object-Centric Motion Segmentation

arXiv 2312.11463 ↗First posted 18 Dec 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

The goal of this paper is to discover, segment, and track independently moving objects in complex visual scenes. Previous approaches have explored the use of optical flow for motion segmentation, leading to imperfect predictions due to partial motion, background distraction, and object articulations and interactions. To address this issue, we introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars, and an object-centric architecture that refines problematic masks based on exemplar information. The model is pre-trained on synthetic data and then adapted to real-world videos in a self-supervised manner, eliminating the need for human annotations. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTubeVOS, SegTrackv2, and FBMS-59. We achieve competitive performance on single-object segmentation, while significantly outperforming existing models on the more challenging problem of multi-object segmentation. Finally, we investigate the benefits of using our model as a prompt for the per-frame Segment Anything Model.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 87 of its 113 references that are in the map. First posted on arXiv 18 Dec 2023.

Builds on

87 of 113 references.

References · 87

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2023Segment Anythingcited by 8,158
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2017Mask R-CNNcited by 7,259
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2018SlowFast Networks for Video Recognitioncited by 1,862
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  11. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  12. 2020Object-Centric Learning with Slot Attentioncited by 865
38 more
  1. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  2. 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
  3. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
  4. 2023Segment Anything in High Qualitycited by 373
  5. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  6. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  7. 2021Conditional Object-Centric Learning from Videocited by 247
  8. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  9. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  10. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  11. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  12. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  13. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  14. 2018Tracking Emerges by Colorizing Videoscited by 166
  15. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 166
  16. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
  17. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  18. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  19. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  20. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  21. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  22. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  23. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  24. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  25. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  26. 2019The 2019 DAVIS Challenge on VOS: Unsupervised Multi-Object Segmentationcited by 95
  27. 2019RVOS: End-to-End Recurrent Network for Video Object Segmentationcited by 88
  28. 2022CenterCLIP: Token Clustering for Efficient Text-Video Retrievalcited by 86
  29. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  30. 2017FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videoscited by 77
  31. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  32. 2016It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videoscited by 71
  33. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  34. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  35. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
  36. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  37. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  38. 2016Learning Motion Patterns in Videoscited by 53

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 4.

11 papers.

  1. 2025Segment Any Motion in Videoscited by 34
  2. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  3. 20254D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimationcited by 8
  4. 2024Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimulicited by 5
  5. 2025More than a Moment: Towards Coherent Sequences of Audio Descriptionscited by 5
  6. 2026CAMotion: A High-Quality Benchmark for Camouflaged Moving Object Detection in the Wildcited by 1
  7. 2026GMOS: Grounding Moving Object Segmentation in 3D Space and Timecited by 1
  8. 2025The Dynamic Prior: Understanding 3D Structures for Casual Dynamic Videoscited by 0
  9. 2025S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentationcited by 0
  10. 2026Temporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditionscited by 0
  11. 2026SUMO: Segment and Track Any Motion with Nonlinear State Space Modelscited by 0