mlsanityData to 14 Oct 2026

Semantic-Aware Fine-Grained Correspondence

arXiv 2207.10456 ↗First posted 21 Jul 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Establishing visual correspondence across images is a challenging and essential task. Recently, an influx of self-supervised methods have been proposed to better learn representations for visual correspondence. However, we find that these methods often fail to leverage semantic information and over-rely on the matching of low-level features. In contrast, human vision is capable of distinguishing between distinct objects as a pretext to tracking. Inspired by this paradigm, we propose to learn semantic-aware fine-grained correspondence. Firstly, we demonstrate that semantic correspondence is implicitly available through a rich set of image-level self-supervised methods. We further design a pixel-level self-supervised learning objective which specifically targets fine-grained correspondence. For downstream tasks, we fuse these two kinds of complementary correspondence representations together, demonstrating that they boost performance synergistically. Our method surpasses previous state-of-the-art self-supervised methods using convolutional networks on a variety of visual correspondence tasks, including video object segmentation, human pose tracking, and human part tracking.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 66 of its 82 references that are in the map. First posted on arXiv 21 Jul 2022.

Builds on

66 of 82 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  12. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  2. 2019Contrastive Multiview Codingcited by 1,301
  3. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  4. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  5. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  6. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  9. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  10. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  11. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  12. 2020Dense Contrastive Learning for Self-Supervised Visual Pre-Trainingcited by 512
  13. 2019Video Object Segmentation using Space-Time Memory Networkscited by 438
  14. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  15. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 383
  16. 2020SCAN: Learning to Classify Images without Labelscited by 352
  17. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  18. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  19. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  20. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  21. 2016One-Shot Video Object Segmentationcited by 235
  22. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  23. 2021Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentationcited by 203
  24. 2019FEELVOS: Fast End-to-End Embedding Learning for Video Object Segmentationcited by 178
  25. 2018Neighbourhood Consensus Networkscited by 174
  26. 2018Tracking Emerges by Colorizing Videoscited by 166
  27. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  28. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  29. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  30. 2017Convolutional neural network architecture for geometric matchingcited by 138
  31. 2020Intriguing Properties of Contrastive Lossescited by 130
  32. 2014Hypercolumns for Object Segmentation and Fine-grained Localizationcited by 128
  33. 2017Online Adaptation of Convolutional Neural Networks for Video Object Segmentationcited by 109
  34. 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
  35. 2017End-to-end representation learning for Correlation Filter based trackingcited by 90
  36. 2016Learning to Track at 100 FPS with Deep Regression Networkscited by 89
  37. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  38. 2019Joint-task Self-supervised Learning for Temporal Correspondencecited by 80

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 9; 2024: 2; 2025: 1.

13 papers.

Sort:

By area:

  1. 2023Emergent Correspondence from Image Diffusioncited by 446
  2. 2023For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equalcited by 22
  3. 2023Single-Stage Visual Query Localization in Egocentric Videoscited by 20
  4. 2023Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videoscited by 20
  5. 2023Deep Learning Technique for Human Parsing: A Survey and Outlookcited by 18
  6. 2024Where's Waldo: Diffusion Features for Personalized Segmentation and Retrievalcited by 18
  7. 2023A Universal Semantic-Geometric Representation for Robotic Manipulationcited by 16
  8. 2022Spatial-then-Temporal Self-Supervised Learning for Video Correspondencecited by 13
  9. 2023Betrayed by Attention: A Simple yet Effective Approach for Self-supervised Video Object Segmentationcited by 10
  10. 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6
  11. 2023Learning Fine-Grained Features for Pixel-wise Video Correspondencescited by 4
  12. 2024Dynamic in Static: Hybrid Visual Correspondence for Self-Supervised Video Object Segmentationcited by 0
1 more
  1. 2025Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervisioncited by 0