mlsanityData to 7 Oct 2026

Watching the World Go By: Representation Learning from Unlabeled Videos

arXiv 2003.07990 ↗First posted 18 Mar 2020cs.CVcited by 59

Not on the map. Why?

Abstract (arXiv)

Recent single image unsupervised representation learning techniques show remarkable success on a variety of tasks. The basic principle in these works is instance discrimination: learning to differentiate between two augmented versions of the same image and a large batch of unrelated images. Networks learn to ignore the augmentation noise and extract semantically meaningful representations. Prior work uses artificial data augmentation techniques such as cropping, and color jitter which can only affect the image in superficial ways and are not aligned with how objects actually change e.g. occlusion, deformation, viewpoint change. In this paper, we argue that videos offer this natural augmentation for free. Videos can provide entirely new views of objects, show deformation, and even connect semantically similar but visually distinct concepts. We propose Video Noise Contrastive Estimation, a method for using unlabeled video to learn strong, transferable single image representations. We demonstrate improvements over recent unsupervised single image techniques, as well as over fully supervised ImageNet pretraining, across a variety of temporal and non-temporal tasks. Code and the Random Related Video Views dataset are available at https://www.github.com/danielgordon10/vince

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 24; 2022: 15; 2023: 8; 2024: 5; 2025: 5; 2026: 2.

59 papers.

Sort:

By area:

  1. 2023Emergent Correspondence from Image Diffusioncited by 437
  2. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  3. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  4. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  5. 2021Divide and Contrast: Self-supervised Learning from Uncurated Datacited by 85
  6. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  7. 2022Video Transformers: A Surveycited by 63
  8. 2021Semi-Supervised Action Recognition with Temporal Contrastive Learningcited by 50
  9. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  10. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 43
  11. 2021SODA10M: A Large-Scale 2D Self/Semi-Supervised Object Detection Dataset for Autonomous Drivingcited by 43
  12. 2021Video Contrastive Learning with Global Contextcited by 40
38 more
  1. 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
  2. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  3. 2021Self-Supervised Representation Learning from Flow Equivariancecited by 28
  4. 2021Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimizationcited by 27
  5. 2022Joint Embedding Predictive Architectures Focus on Slow Featurescited by 24
  6. 2022Motion-Focused Contrastive Learning of Video Representationscited by 23
  7. 2023DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Taskscited by 22
  8. 2021Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptationcited by 21
  9. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  10. 2021Contrastive Learning of Image Representations with Cross-Video Cycle-Consistencycited by 19
  11. 2021Learning to Cut by Watching Moviescited by 18
  12. 2022Sound Localization by Self-Supervised Time Delay Estimationcited by 18
  13. 2021Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learningcited by 16
  14. 2021Contextualized Spatio-Temporal Contrastive Learning with Self-Supervisioncited by 15
  15. 2022Learning from Untrimmed Videos: Self-Supervised Video Representation Learning with Hierarchical Consistencycited by 15
  16. 2022MemSAC: Memory Augmented Sample Consistency for Large Scale Unsupervised Domain Adaptationcited by 15
  17. 2022Time to augment self-supervised visual representation learningcited by 15
  18. 2023ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoderscited by 14
  19. 2022Semantic-Aware Fine-Grained Correspondencecited by 13
  20. 2022Spatial-then-Temporal Self-Supervised Learning for Video Correspondencecited by 13
  21. 2024PooDLe: Pooled and dense self-supervised learning from naturalistic videoscited by 11
  22. 2022Controllable Augmentations for Video Representation Learningcited by 10
  23. 2024Self-supervised visual learning from interactions with objectscited by 10
  24. 2024Self-Supervised Any-Point Tracking by Contrastive Random Walkscited by 10
  25. 2021Characterizing and Improving the Robustness of Self-Supervised Learning through Background Augmentationscited by 8
  26. 2023GRID: A Platform for General Robot Intelligence Developmentcited by 6
  27. 2024Learning text-to-video retrieval from image captioningcited by 6
  28. 2021Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learningcited by 5
  29. 2021Exploring Temporal Granularity in Self-Supervised Video Representation Learningcited by 5
  30. 2024Learning Object Semantic Similarity with Self-Supervisioncited by 5
  31. 2025MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learningcited by 5
  32. 2021Towards Tokenized Human Dynamics Representationcited by 3
  33. 2022Pixel-level Correspondence for Self-Supervised Learning from Videocited by 3
  34. 2022Extraneousness-Aware Imitation Learningcited by 3
  35. 2021PLSM: A Parallelized Liquid State Machine for Unintentional Action Detectioncited by 2
  36. 2021PreViTS: Contrastive Pretraining with Video Tracking Supervisioncited by 2
  37. 2023Text Attribute Control via Closed-Loop Disentanglementcited by 2
  38. 2025Self-Supervised Spatial Correspondence Across Modalitiescited by 2

50 most cited of 59 All 59 (JSON)