mlsanityData to 7 Oct 2026

Segmenting Moving Objects via an Object-Centric Layered Representation

arXiv 2207.02206 ↗First posted 5 Jul 2022cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

The objective of this paper is a model that is able to discover, track and segment multiple moving objects in a video. We make four contributions: First, we introduce an object-centric segmentation model with a depth-ordered layer representation. This is implemented using a variant of the transformer architecture that ingests optical flow, where each query vector specifies an object and its layer for the entire video. The model can effectively discover multiple moving objects and handle mutual occlusions; Second, we introduce a scalable pipeline for generating multi-object synthetic training data via layer compositions, that is used to train the proposed model, significantly reducing the requirements for labour-intensive annotations, and supporting Sim2Real generalisation; Third, we conduct thorough ablation studies, showing that the model is able to learn object permanence and temporal shape consistency, and is able to predict amodal segmentation masks; Fourth, we evaluate our model, trained only on synthetic data, on standard video segmentation benchmarks, DAVIS, MoCA, SegTrack, FBMS-59, and achieve state-of-the-art performance among existing methods that do not rely on any manual annotations. With test-time adaptation, we observe further performance boosts.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 53 of its 91 references that are in the map. First posted on arXiv 5 Jul 2022.

Builds on

53 of 91 references.

References · 53

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2017Mask R-CNNcited by 7,259
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  8. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  9. 2020Object-Centric Learning with Slot Attentioncited by 865
  10. 2019Video Object Segmentation using Space-Time Memory Networkscited by 436
  11. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  12. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
38 more
  1. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  2. 2021Conditional Object-Centric Learning from Videocited by 247
  3. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  4. 2016One-Shot Video Object Segmentationcited by 233
  5. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  6. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  7. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  8. 2018Tracking Emerges by Colorizing Videoscited by 166
  9. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  10. 2020See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networkscited by 138
  11. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  12. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  13. 2018"Double-DIP": Unsupervised Image Decomposition via Coupled Deep-Image-Priorscited by 114
  14. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  15. 2015Semantic Amodal Segmentationcited by 102
  16. 2020Self-Supervised Scene De-occlusioncited by 96
  17. 2017Video Object Segmentation Without Temporal Informationcited by 93
  18. 2017SegFlow: Joint Learning for Video Object Segmentation and Optical Flowcited by 92
  19. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80
  20. 2017FusionSeg: Learning to combine motion and appearance for fully automatic segmention of generic objects in videoscited by 77
  21. 2019Sim2real transfer learning for 3D human pose estimation: motion to the rescuecited by 75
  22. 2019Unsupervised Moving Object Detection via Contextual Information Separationcited by 72
  23. 2016It's Moving! A Probabilistic Model for Causal Motion Segmentation in Moving Camera Videoscited by 71
  24. 2016Amodal Instance Segmentationcited by 63
  25. 2020Betrayed by Motion: Camouflaged Object Discovery via Motion Segmentationcited by 62
  26. 2020Making a Case for 3D Convolutions for Object Segmentation in Videoscited by 56
  27. 2016Learning Motion Patterns in Videoscited by 53
  28. 2019Towards Segmenting Anything That Movescited by 51
  29. 2020Layered Neural Rendering for Retiming People in Videocited by 51
  30. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  31. 2019Anchor Diffusion for Unsupervised Video Object Segmentationcited by 50
  32. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  33. 2020Amodal Segmentation Based on Visible Region Segmentation and Shape Priorcited by 42
  34. 2017Learning to Segment Moving Objectscited by 39
  35. 2018Object Discovery in Videos as Foreground Motion Clusteringcited by 32
  36. 2020Amodal Segmentation through Out-of-Task and Out-of-Distribution Generalization with a Bayesian Modelcited by 29
  37. 2018The Visual Centrifuge: Model-Free Layered Video Representationscited by 23
  38. 2021D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videoscited by 20

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 19; 2024: 16; 2025: 16; 2026: 10.

63 papers.

Sort:

By area:

  1. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 88
  2. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  3. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  4. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  5. 2023Advances in Deep Concealed Scene Understandingcited by 47
  6. 2023Amodal Ground Truth and Completion in the Wildcited by 43
  7. 2024A Survey of Camouflaged Object Detection and Beyondcited by 37
  8. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 36
  9. 2025Segment Any Motion in Videoscited by 34
  10. 2024Moving Object Segmentation: All You Need Is SAM (and Flow)cited by 30
  11. 2023Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videoscited by 20
  12. 2024RoMo: Robust Motion Segmentation Improves Structure from Motioncited by 20
38 more
  1. 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
  2. 2023Bootstrapping Objectness from Videos by Relaxed Common Fate and Visual Groupingcited by 15
  3. 2022Motion-inductive Self-supervised Object Discovery in Videoscited by 14
  4. 2023The Making and Breaking of Camouflagecited by 14
  5. 2025Learning segmentation from point trajectoriescited by 14
  6. 2023Appearance-Based Refinement for Object-Centric Motion Segmentationcited by 11
  7. 2024Explicit Motion Handling and Interactive Prompting for Video Camouflaged Object Detectioncited by 11
  8. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  9. 2023Event-Free Moving Object Segmentation from Moving Ego Vehiclecited by 8
  10. 2023LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-trainingcited by 8
  11. 2024Rethinking Image-to-Video Adaptation: An Object-centric Perspectivecited by 8
  12. 20254D-VGGT: A General Foundation Model with SpatioTemporal Awareness for Dynamic Scene Geometry Estimationcited by 8
  13. 2023Amodal Optical Flowcited by 7
  14. 2024Out of the Room: Generalizing Event-Based Dynamic Motion Segmentation for Complex Scenescited by 7
  15. 2024Reasoning-Enhanced Object-Centric Learning for Videoscited by 7
  16. 2025CamSAM2: Segment Anything Accurately in Camouflaged Videoscited by 7
  17. 2024On Moving Object Segmentation from Monocular Video with Transformerscited by 6
  18. 2024Made to Order: Discovering monotonic temporal changes via self-supervised video orderingcited by 5
  19. 2024Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimulicited by 5
  20. 2024Dynamic Scene Understanding through Object-Centric Voxelization and Neural Renderingcited by 4
  21. 2023DynaVol: Unsupervised Learning for Dynamic Scenes through Object-Centric Voxelizationcited by 3
  22. 2023FODVid: Flow-guided Object Discovery in Videoscited by 3
  23. 2023Segmenting the motion components of a video: A long-term unsupervised modelcited by 2
  24. 2024Motion Segmentation for Neuromorphic Aerial Surveillancecited by 2
  25. 2024Fast Sprite Decomposition from Animated Graphicscited by 2
  26. 2025FlowCut: Unsupervised Video Instance Segmentation via Temporal Mask Matchingcited by 2
  27. 2024CORP: A Multi-Modal Dataset for Campus-Oriented Roadside Perception Taskscited by 1
  28. 2024Decomposition Betters Tracking Everything Everywherecited by 1
  29. 2025AutoQ-VIS: Improving Unsupervised Video Instance Segmentation via Automatic Quality Assessmentcited by 1
  30. 2025NeoWorld: Neural Simulation of Explorable Virtual Worlds via Progressive 3D Unfoldingcited by 1
  31. 2026CAMotion: A High-Quality Benchmark for Camouflaged Moving Object Detection in the Wildcited by 1
  32. 2026GMOS: Grounding Moving Object Segmentation in 3D Space and Timecited by 1
  33. 2022Improving Unsupervised Video Object Segmentation with Motion-Appearance Synergycited by 0
  34. 2023Divided Attention: Unsupervised Multi-Object Discovery with Contextually Separated Slotscited by 0
  35. 2024Self-supervised Video Instance Segmentation Can Boost Geographic Entity Alignment in Historical Mapscited by 0
  36. 2025xMOD: Cross-Modal Distillation for 2D/3D Multi-Object Discovery from 2D motioncited by 0
  37. 2025Object Concepts Emerge from Motioncited by 0
  38. 2025Layered Motion Fusion: Lifting Motion Segmentation to 3D in Egocentric Videoscited by 0

50 most cited of 63 All 63 (JSON)