# Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels

arXiv: https://arxiv.org/abs/2506.05312 · HTML: https://mlsanity.com/paper/2506.05312/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 69 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 58)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Structured 3D Latents for Scalable and Versatile 3D Generation](https://mlsanity.com/paper/2412.01506/) (2024)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [TAP-Vid: A Benchmark for Tracking Any Point in a Video](https://mlsanity.com/paper/2211.03726/) (2022)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence](https://mlsanity.com/paper/2305.14334/) (2023)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [AiATrack: Attention in Attention for Transformer Visual Tracking](https://mlsanity.com/paper/2207.09603/) (2022)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [End-to-end weakly-supervised semantic alignment](https://mlsanity.com/paper/1712.06861/) (2017)
- [DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision](https://mlsanity.com/paper/2105.06464/) (2021)
- [COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editing](https://mlsanity.com/paper/2406.08850/) (2024)
- [SCNet: Learning Semantic Correspondence](https://mlsanity.com/paper/1705.04043/) (2017)
- [Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models](https://mlsanity.com/paper/2412.18605/) (2024)
- [CleanDIFT: Diffusion Features without Noise](https://mlsanity.com/paper/2412.03439/) (2024)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)

## Built on by (most cited first, 13 of 13)

- [Attention (as Discrete-Time Markov) Chains](https://mlsanity.com/paper/2507.17657/) (2025)
- [GECO: Geometrically Consistent Embedding with Lightspeed Inference](https://mlsanity.com/paper/2508.00746/) (2025)
- [Towards Geometry-Grounded Dense Semantic Matching with VGGT Priors](https://mlsanity.com/paper/2509.21263/) (2025)
- [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://mlsanity.com/paper/2604.18267/) (2026)
- [Shape-of-You: Fused Gromov-Wasserstein Optimal Transport for Semantic Correspondence in-the-Wild](https://mlsanity.com/paper/2603.11618/) (2026)
- [Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence](https://mlsanity.com/paper/2605.30093/) (2026)
- [PASR: Pose-Aware 3D Shape Retrieval from Occluded Single Views](https://mlsanity.com/paper/2604.22658/) (2026)
- [SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images](https://mlsanity.com/paper/2605.27938/) (2026)
- [Category-Level 3D Correspondence in Camera Space via Morphable Object Priors](https://mlsanity.com/paper/2605.28257/) (2026)
- [Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects](https://mlsanity.com/paper/2605.28270/) (2026)
- [SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models](https://mlsanity.com/paper/2605.31597/) (2026)
- [Emergence of a Shared Canonical Object Frame from In-the-Wild Videos](https://mlsanity.com/paper/2606.30058/) (2026)
- [ImCorr: Sub-pixel Semantic Correspondence via Implicit Feature Decoding](https://mlsanity.com/paper/2609.29193/) (2026)

## Abstract (arXiv)

Finding correspondences between semantically similar points across images and object instances is one of the everlasting challenges in computer vision. While large pre-trained vision models have recently been demonstrated as effective priors for semantic matching, they still suffer from ambiguities for symmetric objects or repeated object parts. We propose improving semantic correspondence estimation through 3D-aware pseudo-labeling. Specifically, we train an adapter to refine off-the-shelf features using pseudo-labels obtained via 3D-aware chaining, filtering wrong labels through relaxed cyclic consistency, and 3D spherical prototype mapping constraints. While reducing the need for dataset-specific annotations compared to prior work, we establish a new state-of-the-art on SPair-71k, achieving an absolute gain of over 4% and of over 7% compared to methods with similar supervision requirements. The generality of our proposed approach simplifies the extension of training to other data sources, which we demonstrate in our experiments.