mlsanityData to 14 Oct 2026

Object Concepts Emerge from Motion

arXiv 2505.21635 ↗First posted 27 May 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Object concepts play a foundational role in human visual cognition, enabling perception, memory, and interaction in the physical world. Inspired by findings in developmental neuroscience - where infants are shown to acquire object understanding through observation of motion - we propose a biologically inspired framework for learning object-centric visual representations in an unsupervised manner. Our key insight is that motion boundary serves as a strong signal for object-level grouping, which can be used to derive pseudo instance supervision from raw videos. Concretely, we generate motion-based instance masks using off-the-shelf optical flow and clustering algorithms, and use them to train visual encoders via contrastive learning. Our framework is fully label-free and does not rely on camera calibration, making it scalable to large-scale unstructured video data. We evaluate our approach on three downstream tasks spanning both low-level (monocular depth estimation) and high-level (3D object detection and occupancy prediction) vision. Our models outperform previous supervised and self-supervised baselines and demonstrate strong generalization to unseen scenes. These results suggest that motion-induced object representations offer a compelling alternative to existing vision foundation models, capturing a crucial but overlooked level of abstraction: the visual instance. The corresponding code will be released upon paper acceptance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 76 of its 117 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

76 of 117 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2023Segment Anythingcited by 8,234
  8. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  12. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
38 more
  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  3. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  5. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  6. 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,085
  7. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  9. 2024Depth Anything V2cited by 1,422
  10. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  11. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  13. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  14. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  15. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  16. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  17. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  18. 2020Object-Centric Learning with Slot Attentioncited by 876
  19. 2023Vision Transformers Need Registerscited by 871
  20. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  21. 2019Panoptic Feature Pyramid Networkscited by 645
  22. 2018Born Again Neural Networkscited by 546
  23. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  24. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  25. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  26. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  27. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  28. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  29. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  30. 2022BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizerscited by 303
  31. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  32. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  33. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  34. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 230
  35. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  36. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 167
  37. 2022CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completioncited by 160
  38. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 153

50 most cited of 76 All 76 (JSON)

Built on by

Not cited yet.