mlsanityData to 7 Oct 2026

Space-Time Correspondence as a Contrastive Random Walk

arXiv 2006.14613 ↗First posted 25 Jun 2020cs.CV cs.LG eess.IVcited by 239

Not on the map. Why?

Abstract (arXiv)

This paper proposes a simple self-supervised approach for learning a representation for visual correspondence from raw video. We cast correspondence as prediction of links in a space-time graph constructed from video. In this graph, the nodes are patches sampled from each frame, and nodes adjacent in time can share a directed edge. We learn a representation in which pairwise similarity defines transition probability of a random walk, so that long-range correspondence is computed as a walk along the graph. We optimize the representation to place high probability along paths of similarity. Targets for learning are formed without supervision, by cycle-consistency: the objective is to maximize the likelihood of returning to the initial node when walking along a graph constructed from a palindrome of frames. Thus, a single path-level constraint implicitly supervises chains of intermediate comparisons. When used as a similarity metric without adaptation, the learned representation outperforms the self-supervised state-of-the-art on label propagation tasks involving objects, semantic parts, and pose. Moreover, we demonstrate that a technique we call edge dropout, as well as self-supervised adaptation at test-time, further improve transfer for object-centric correspondence.

Key facts. Cited by 239 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 51; 2022: 50; 2023: 46; 2024: 41; 2025: 38; 2026: 13.

239 papers.

Sort:

By area:

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  4. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  5. 2023Emergent Correspondence from Image Diffusioncited by 437
  6. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 315
  7. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  8. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  9. 2021Conditional Object-Centric Learning from Videocited by 247
  10. 2022Masked Visual Pre-training for Motor Controlcited by 246
  11. 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
  12. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
38 more
  1. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  2. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
  3. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  4. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  5. 2023Tracking Everything Everywhere All at Oncecited by 190
  6. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  7. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 168
  8. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
  9. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  10. 2021Towards Long-Form Video Understandingcited by 146
  11. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  12. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  13. 2021How to represent part-whole hierarchies in a neural networkcited by 123
  14. 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 122
  15. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 115
  16. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  17. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  18. 2021SiT: Self-supervised vIsion Transformercited by 99
  19. 2022Towards Grand Unification of Object Trackingcited by 93
  20. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  21. 2023Segment Anything Meets Point Trackingcited by 81
  22. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  23. 2023Siamese Masked Autoencoderscited by 75
  24. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  25. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
  26. 2021Do Different Tracking Tasks Require Different Appearance Models?cited by 62
  27. 2021Joint Inductive and Transductive Learning for Video Object Segmentationcited by 61
  28. 2022Patch-level Representation Learning for Self-supervised Vision Transformerscited by 60
  29. 2025MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenescited by 59
  30. 2023A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyondcited by 55
  31. 2022Mix and Localize: Localizing Sound Sources in Mixturescited by 54
  32. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  33. 2022CATs++: Boosting Cost Aggregation with Convolutions and Transformerscited by 53
  34. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  35. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  36. 2021Revisiting Contrastive Methods for Unsupervised Learning of Visual Representationscited by 48
  37. 2022Breaking the "Object" in Video Object Segmentationcited by 48
  38. 2021Adversarial Attacks are Reversible with Natural Supervisioncited by 47

50 most cited of 239 All 239 (JSON)