mlsanityData to 7 Oct 2026

Learning Features by Watching Objects Move

arXiv 1612.06370 ↗First posted 19 Dec 2016cs.CV cs.AI cs.LGcited by 138

Not on the map. Why?

Abstract (arXiv)

This paper presents a novel yet intuitive approach to unsupervised feature learning. Inspired by the human visual system, we explore whether low-level motion-based grouping cues can be used to learn an effective visual representation. Specifically, we use unsupervised motion-based segmentation on videos to obtain segments, which we use as 'pseudo ground truth' to train a convolutional network to segment objects from a single frame. Given the extensive evidence that motion plays a key role in the development of the human visual system, we hope that this straightforward approach to unsupervised learning will be more effective than cleverly designed 'pretext' tasks studied in the literature. Indeed, our extensive experiments show that this is the case. When used for transfer learning on object detection, our representation significantly outperforms previous unsupervised approaches across multiple settings, especially when training data for the target task is scarce.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 40; 2022: 37; 2023: 30; 2024: 12; 2025: 10; 2026: 8.

138 papers.

Sort:

By area:

  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  2. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  3. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  4. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  5. 2023A Cookbook of Self-Supervised Learningcited by 248
  6. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  7. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  8. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  9. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  10. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  11. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  12. 2019Imitation Learning from Imperfect Demonstrationcited by 112
38 more
  1. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  2. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  3. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  4. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  5. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  6. 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
  7. 2023Siamese Masked Autoencoderscited by 75
  8. 2022Masked Image Modeling Advances 3D Medical Image Analysiscited by 71
  9. 2023Diffusion Models as Masked Autoencoderscited by 71
  10. 2022Masked Frequency Modeling for Self-Supervised Visual Pre-Trainingcited by 70
  11. 2024Improving 2D Feature Representations by 3D-Aware Fine-Tuningcited by 70
  12. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 59
  13. 2021Data-Efficient Instance Generation from Instance Discriminationcited by 58
  14. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  15. 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
  16. 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
  17. 2022Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspectivecited by 46
  18. 2021The Emergence of Objectness: Learning Zero-Shot Segmentation from Videoscited by 45
  19. 2022Discovering Objects that Can Movecited by 44
  20. 2022Learning to Detect Mobile Objects from LiDAR Scans Without Labelscited by 44
  21. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 43
  22. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  23. 2021Contrasting Contrastive Self-Supervised Representation Learning Pipelinescited by 37
  24. 2022On the Power of Foundation Modelscited by 37
  25. 2022The Challenges of Continuous Self-Supervised Learningcited by 36
  26. 2023Neural Preset for Color Style Transfercited by 36
  27. 2021Shot Contrastive Self-Supervised Learning for Scene Boundary Detectioncited by 35
  28. 2022SSL-Lanes: Self-Supervised Learning for Motion Forecasting in Autonomous Drivingcited by 31
  29. 2022MeshMAE: Masked Autoencoders for 3D Mesh Data Analysiscited by 31
  30. 2021Jigsaw Clustering for Unsupervised Visual Representation Learningcited by 30
  31. 2024Scaling 4D Representationscited by 30
  32. 2021Unsupervised Object Learning via Common Fatecited by 29
  33. 2021Weakly Supervised Video Salient Object Detectioncited by 28
  34. 2024Masked Image Modeling: A Surveycited by 28
  35. 2022Efficient Self-supervised Vision Pretraining with Local Masked Reconstructioncited by 27
  36. 2021Dense Unsupervised Learning for Video Segmentationcited by 26
  37. 2022Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learningcited by 26
  38. 2021Self-supervised Motion Learning from Static Imagescited by 21

50 most cited of 138 All 138 (JSON)