# Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps

arXiv: https://arxiv.org/abs/2312.13216 · HTML: https://mlsanity.com/paper/2312.13216/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 63 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [Shape and Viewpoint without Keypoints](https://mlsanity.com/paper/2007.10982/) (2020)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields](https://mlsanity.com/paper/2203.01913/) (2022)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [Universal Correspondence Network](https://mlsanity.com/paper/1606.03558/) (2016)
- [Continuous Surface Embeddings](https://mlsanity.com/paper/2011.12438/) (2020)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [Canonical Surface Mapping via Geometric Cycle Consistency](https://mlsanity.com/paper/1907.10043/) (2019)
- [Proposal Flow](https://mlsanity.com/paper/1511.05065/) (2015)
- [Learning to Compose Hypercolumns for Visual Correspondence](https://mlsanity.com/paper/2007.10587/) (2020)
- [SCNet: Learning Semantic Correspondence](https://mlsanity.com/paper/1705.04043/) (2017)
- [Teaching Matters: Investigating the Role of Supervision in Vision Transformers](https://mlsanity.com/paper/2212.03862/) (2022)
- [Unsupervised Learning of Landmarks by Descriptor Vector Exchange](https://mlsanity.com/paper/1908.06427/) (2019)
- [Multi-scale Matching Networks for Semantic Correspondence](https://mlsanity.com/paper/2108.00211/) (2021)

## Built on by (most cited first, 30 of 34)

- [DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video](https://mlsanity.com/paper/2403.14548/) (2024)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Distilling Spectral Graph for Object-Context Aware Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2411.17150/) (2024)
- [DepthCues: Evaluating Monocular Depth Perception in Large Vision Models](https://mlsanity.com/paper/2411.17385/) (2024)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Exploring Temporally-Aware Features for Point Tracking](https://mlsanity.com/paper/2501.12218/) (2025)
- [Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels](https://mlsanity.com/paper/2506.05312/) (2025)
- [Click to Grasp: Zero-Shot Precise Manipulation via Visual Diffusion Descriptors](https://mlsanity.com/paper/2403.14526/) (2024)
- [Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space](https://mlsanity.com/paper/2504.21749/) (2025)
- [Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose Estimation](https://mlsanity.com/paper/2503.13926/) (2025)
- [SemAlign3D: Semantic Correspondence between RGB-Images through Aligning 3D Object-Class Representations](https://mlsanity.com/paper/2503.22462/) (2025)
- [SpaceJAM: a Lightweight and Regularization-free Method for Fast Joint Alignment of Images](https://mlsanity.com/paper/2407.11850/) (2024)
- [Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence](https://mlsanity.com/paper/2506.08220/) (2025)
- [GECO: Geometrically Consistent Embedding with Lightspeed Inference](https://mlsanity.com/paper/2508.00746/) (2025)
- [Surface-Aware Distilled 3D Semantic Features](https://mlsanity.com/paper/2503.18254/) (2025)
- [Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2503.19914/) (2025)
- [Towards Geometry-Grounded Dense Semantic Matching with VGGT Priors](https://mlsanity.com/paper/2509.21263/) (2025)
- [MARCO: Navigating the Unseen Space of Semantic Correspondence](https://mlsanity.com/paper/2604.18267/) (2026)
- [Attentive Feature Aggregation or: How Policies Learn to Stop Worrying about Robustness and Attend to Task-Relevant Visual Cues](https://mlsanity.com/paper/2511.10762/) (2025)
- [PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks](https://mlsanity.com/paper/2403.11743/) (2024)
- [Latent Diffusion U-Net Representations Contain Positional Embeddings and Anomalies](https://mlsanity.com/paper/2504.07008/) (2025)
- [Zero-shot Inexact CAD Model Alignment from a Single Image](https://mlsanity.com/paper/2507.03292/) (2025)
- [FastJAM: a Fast Joint Alignment Model for Images](https://mlsanity.com/paper/2510.22842/) (2025)
- [Shape-of-You: Fused Gromov-Wasserstein Optimal Transport for Semantic Correspondence in-the-Wild](https://mlsanity.com/paper/2603.11618/) (2026)
- [Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence](https://mlsanity.com/paper/2605.30093/) (2026)
- [Gromov Wasserstein Optimal Transport for Semantic Correspondences](https://mlsanity.com/paper/2602.03105/) (2026)
- [Cross-Instance Gaussian Splatting Registration via Geometry-Aware Feature-Guided Alignment](https://mlsanity.com/paper/2603.21936/) (2026)
- [SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Images](https://mlsanity.com/paper/2605.27938/) (2026)
- [Category-Level 3D Correspondence in Camera Space via Morphable Object Priors](https://mlsanity.com/paper/2605.28257/) (2026)
- [Every9D-21M: Large-Scale Real-World 9D Canonicalization of Everyday Objects](https://mlsanity.com/paper/2605.28270/) (2026)

## Abstract (arXiv)

Recent progress in self-supervised representation learning has resulted in models that are capable of extracting image features that are not only effective at encoding image level, but also pixel-level, semantics. These features have been shown to be effective for dense visual semantic correspondence estimation, even outperforming fully-supervised methods. Nevertheless, current self-supervised approaches still fail in the presence of challenging image characteristics such as symmetries and repeated parts. To address these limitations, we propose a new approach for semantic correspondence estimation that supplements discriminative self-supervised features with 3D understanding via a weak geometric spherical prior. Compared to more involved 3D pipelines, our model only requires weak viewpoint information, and the simplicity of our spherical representation enables us to inject informative geometric priors into the model during training. We propose a new evaluation metric that better accounts for repeated part and symmetry-induced mistakes. We present results on the challenging SPair-71k dataset, where we show that our approach demonstrates is capable of distinguishing between symmetric views and repeated parts across many object categories, and also demonstrate that we can generalize to unseen classes on the AwA dataset.