mlsanityData to 7 Oct 2026

Joint-task Self-supervised Learning for Temporal Correspondence

arXiv 1909.11895 ↗First posted 26 Sep 2019cs.CVcited by 80

Not on the map. Why?

Abstract (arXiv)

This paper proposes to learn reliable dense correspondence from videos in a self-supervised manner. Our learning process integrates two highly related tasks: tracking large image regions and establishing fine-grained pixel-level associations between consecutive video frames. We exploit the synergy between both tasks through a shared inter-frame affinity matrix, which simultaneously models transitions between video frames at both the region- and pixel-levels. While region-level localization helps reduce ambiguities in fine-grained matching by narrowing down search regions; fine-grained matching provides bottom-up features to facilitate region-level localization. Our method outperforms the state-of-the-art self-supervised methods on a variety of visual correspondence tasks, including video-object and part-segmentation propagation, keypoint tracking, and object tracking. Our self-supervised method even surpasses the fully-supervised affinity feature representation obtained from a ResNet-18 pre-trained on the ImageNet.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 20; 2023: 20; 2024: 7; 2025: 9; 2026: 2.

80 papers.

Sort:

By area:

  1. 2023Emergent Correspondence from Image Diffusioncited by 437
  2. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  3. 2021Conditional Object-Centric Learning from Videocited by 247
  4. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  5. 2023Tracking Everything Everywhere All at Oncecited by 190
  6. 2021A Survey on Deep Learning Technique for Video Segmentationcited by 132
  7. 2021Region Similarity Representation Learningcited by 96
  8. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  9. 2023Siamese Masked Autoencoderscited by 75
  10. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  11. 2022Segmenting Moving Objects via an Object-Centric Layered Representationcited by 63
  12. 2021Do Different Tracking Tasks Require Different Appearance Models?cited by 62
38 more
  1. 2022ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detectioncited by 43
  2. 2023Is ImageNet worth 1 video? Learning strong image encoders from 1 long unlabelled videocited by 40
  3. 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
  4. 2023VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentationcited by 36
  5. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  6. 2021Self-Supervised Representation Learning from Flow Equivariancecited by 28
  7. 2021Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimizationcited by 27
  8. 2021Dense Unsupervised Learning for Video Segmentationcited by 26
  9. 2022Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learningcited by 26
  10. 2021Learning to Associate Every Segment for Video Panoptic Segmentationcited by 25
  11. 2021Self-Supervised Multi-Object Tracking with Cross-Input Consistencycited by 25
  12. 2022Static and Dynamic Concepts for Self-supervised Video Representation Learningcited by 23
  13. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  14. 2023DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Taskscited by 22
  15. 2021Learning to Track Objects from Unlabeled Videoscited by 21
  16. 2023Boosting Video Object Segmentation via Space-time Correspondence Learningcited by 21
  17. 2021Unsupervised Visual Representation Learning by Tracking Patches in Videocited by 20
  18. 2022Partially Does It: Towards Scene-Level FG-SBIR with Partial Inputcited by 20
  19. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  20. 2023Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videoscited by 20
  21. 2021Learning to Track Instances without Video Annotationscited by 18
  22. 2023Deep Learning Technique for Human Parsing: A Survey and Outlookcited by 18
  23. 2022CoordGAN: Self-Supervised Dense Correspondences Emerge from GANscited by 17
  24. 2021Modelling Neighbor Relation in Joint Space-Time Graph for Video Correspondence Learningcited by 16
  25. 2023Unified Mask Embedding and Correspondence Learning for Self-Supervised Video Segmentationcited by 16
  26. 2022Dual Contrastive Learning for Spatio-temporal Representationcited by 15
  27. 2022Motion-inductive Self-supervised Object Discovery in Videoscited by 14
  28. 2022Semantic-Aware Fine-Grained Correspondencecited by 13
  29. 2022Spatial-then-Temporal Self-Supervised Learning for Video Correspondencecited by 13
  30. 2024Refining Pre-Trained Motion Modelscited by 12
  31. 2025Predicting Video Slot Attention Queries from Random Slot-Feature Pairscited by 12
  32. 2021Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagationcited by 11
  33. 2021Learning Contrastive Representation for Semantic Correspondencecited by 11
  34. 2023Appearance-Based Refinement for Object-Centric Motion Segmentationcited by 11
  35. 2022Controllable Augmentations for Video Representation Learningcited by 10
  36. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  37. 2023No More Shortcuts: Realizing the Potential of Temporal Self-Supervisioncited by 10
  38. 2024Visual Representation Learning with Stochastic Frame Predictioncited by 10

50 most cited of 80 All 80 (JSON)