mlsanityData to 14 Oct 2026

Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similarities

arXiv 2306.04829 ↗First posted 7 Jun 2023cs.CV cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

Unsupervised video-based object-centric learning is a promising avenue to learn structured representations from large, unlabeled video collections, but previous approaches have only managed to scale to real-world datasets in restricted domains. Recently, it was shown that the reconstruction of pre-trained self-supervised features leads to object-centric representations on unconstrained real-world image datasets. Building on this approach, we propose a novel way to use such pre-trained features in the form of a temporal feature similarity loss. This loss encodes semantic and temporal correlations between image patches and is a natural way to introduce a motion bias for object discovery. We demonstrate that this loss leads to state-of-the-art performance on the challenging synthetic MOVi datasets. When used in combination with the feature reconstruction loss, our model is the first object-centric video model that scales to unconstrained video datasets such as YouTube-VIS.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 7 Jun 2023.

Builds on

48 of 58 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  6. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  7. 2020Object-Centric Learning with Slot Attentioncited by 876
  8. 2019Video Instance Segmentationcited by 393
  9. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  10. 2022Kubric: A scalable dataset generatorcited by 381
  11. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  12. 2022Masked Siamese Networks for Label-Efficient Learningcited by 302
36 more
  1. 2021Conditional Object-Centric Learning from Videocited by 248
  2. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  3. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  4. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  5. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  6. 2021Illiterate DALL-E Learns to Composecited by 147
  7. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  8. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  9. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  10. 2016Discrete Variational Autoencoderscited by 125
  11. 2017Neural Expectation Maximizationcited by 123
  12. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  13. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  14. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  15. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  16. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  17. 2015Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generationcited by 91
  18. 2022Object Scene Representation Transformercited by 90
  19. 2023Object-Centric Slot Diffusioncited by 90
  20. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  21. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  22. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
  23. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  24. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  25. 2022Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiationcited by 46
  26. 2020Self-supervised Visual Reinforcement Learning with Object-centric Representationscited by 44
  27. 2022Discovering Objects that Can Movecited by 44
  28. 2021Unsupervised Object Learning via Common Fatecited by 29
  29. 2020Benchmarking Unsupervised Object Representations for Video Sequencescited by 27
  30. 2023Object Discovery from Motion-Guided Tokenscited by 25
  31. 2022Learning What and Where: Disentangling Location and Identity Tracking Without Supervisioncited by 21
  32. 2022Compositional Multi-Object Reinforcement Learning with Linear Relation Networkscited by 19
  33. 2021Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domainscited by 18
  34. 2023Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learningcited by 17
  35. 2022Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Imagescited by 16
  36. 2022Differentiable Mathematical Programming for Object-Centric Representation Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 19; 2025: 26; 2026: 31.

78 papers.

Sort:

By area:

  1. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  2. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  3. 2024Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imaginationcited by 45
  4. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 31
  5. 2026Causal-JEPA: Learning World Models through Object-Level Latent Maskingcited by 29
  6. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  7. 2025Object-Centric World Model for Language-Guided Manipulationcited by 16
  8. 2024Near, far: Patch-ordering enhances vision foundation models' scene understandingcited by 15
  9. 2025Learning segmentation from point trajectoriescited by 15
  10. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 14
  11. 2025Slot Attention with Re-Initialization and Self-Distillationcited by 14
  12. 2024Causal Action Influence Aware Counterfactual Data Augmentationcited by 12
38 more
  1. 2024SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fieldscited by 12
  2. 2025CTRL-O: Language-Controllable Object-Centric Visual Representation Learningcited by 12
  3. 2025Predicting Video Slot Attention Queries from Random Slot-Feature Pairscited by 12
  4. 2025Vector-Quantized Vision Foundation Models for Object-Centric Learningcited by 11
  5. 2025MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learningcited by 11
  6. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  7. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  8. 2025Object-Centric Latent Action Learningcited by 9
  9. 2024Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discoverycited by 8
  10. 2024EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generationcited by 8
  11. 2024Multi-Scale Fusion for Object Representationcited by 7
  12. 2026Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusioncited by 7
  13. 2025Object-Centric Representations Improve Policy Generalization in Robot Manipulationcited by 6
  14. 2025Smoothing Slot Attention Iterations and Recurrencescited by 6
  15. 2024Flow Snapshot Neurons in Action: Deep Neural Networks Generalize to Biological Motion Perceptioncited by 5
  16. 2024Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biasescited by 5
  17. 2024Grouped Discrete Representation for Object-Centric Learningcited by 5
  18. 2025MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learningcited by 5
  19. 2026From Vicious to Virtuous Cycles: Synergistic Representation Learning for Unsupervised Video Object-Centric Learningcited by 5
  20. 2025Slot-BERT: Self-supervised Object Discovery in Surgical Videocited by 4
  21. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  22. 2025unMORE: Unsupervised Multi-Object Segmentation via Center-Boundary Reasoningcited by 4
  23. 2025Motion-Refined DINOSAUR for Unsupervised Multi-Object Discoverycited by 4
  24. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  25. 2025Is an object-centric representation beneficial for robotic manipulation ?cited by 3
  26. 2026Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignmentcited by 3
  27. 2026Reconstruction-Guided Slot Curriculum: Addressing Object Over-Fragmentation in Video Object-Centric Learningcited by 3
  28. 2024VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAEcited by 2
  29. 2024Parallelized Spatiotemporal Bindingcited by 2
  30. 2024Grouped Discrete Representation Guides Object-Centric Learningcited by 2
  31. 2025Looking Locally: Object-Centric Vision Transformers as Foundation Models for Efficient Segmentationcited by 2
  32. 2025Future Slot Prediction for Unsupervised Object Discovery in Surgical Videocited by 2
  33. 2025UNO: Unifying One-stage Video Scene Graph Generation via Object-Centric Visual Representation Learningcited by 2
  34. 2026Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learningcited by 2
  35. 2026TSA: Temporal Slot Activation for Persistent Object-Centric Video Representationcited by 2
  36. 2026Selective Synergistic Learning for Video Object-Centric Learningcited by 2
  37. 2026Rethinking Object-Centric Representations for Video Dynamics Modelingcited by 2
  38. 2024Organized Grouped Discrete Representation for Object-Centric Learningcited by 1

50 most cited of 78 All 78 (JSON)