# Geometry Matters: 3D Foundation Priors for Learning Semantic Correspondence

arXiv: https://arxiv.org/abs/2605.30093 · HTML: https://mlsanity.com/paper/2605.30093/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 28 May 2026.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 34)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence](https://mlsanity.com/paper/2305.14334/) (2023)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence](https://mlsanity.com/paper/2003.14286/) (2020)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [PARTFIELD: Learning 3D Feature Fields for Part Segmentation and Beyond](https://mlsanity.com/paper/2504.11451/) (2025)
- [Learning Dense Correspondence via 3D-guided Cycle Consistency](https://mlsanity.com/paper/1604.05383/) (2016)
- [Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence](https://mlsanity.com/paper/2311.17034/) (2023)
- [Canonical Surface Mapping via Geometric Cycle Consistency](https://mlsanity.com/paper/1907.10043/) (2019)
- [Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features](https://mlsanity.com/paper/2311.17024/) (2023)
- [Proposal Flow: Semantic Correspondences from Object Proposals](https://mlsanity.com/paper/1703.07144/) (2017)
- [SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching](https://mlsanity.com/paper/2310.17569/) (2023)
- [Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps](https://mlsanity.com/paper/2312.13216/) (2023)
- [DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Demo](https://mlsanity.com/paper/2412.05268/) (2024)
- [Orient Anything V2: Unifying Orientation and Rotation Understanding](https://mlsanity.com/paper/2601.05573/) (2026)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [Semi-Supervised Learning of Semantic Correspondence with Pseudo-Labels](https://mlsanity.com/paper/2203.16038/) (2022)
- [Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels](https://mlsanity.com/paper/2506.05312/) (2025)
- [GS-Pose: Category-Level Object Pose Estimation via Geometric and Semantic Correspondence](https://mlsanity.com/paper/2311.13777/) (2023)
- [SHIC: Shape-Image Correspondences with no Keypoint Supervision](https://mlsanity.com/paper/2407.18907/) (2024)
- [MATCHA:Towards Matching Anything](https://mlsanity.com/paper/2501.14945/) (2025)
- [NOVUM: Neural Object Volumes for Robust Object Classification](https://mlsanity.com/paper/2305.14668/) (2023)
- [Common3D: Self-Supervised Learning of 3D Morphable Models for Common Objects in Neural Feature Space](https://mlsanity.com/paper/2504.21749/) (2025)

## Built on by (most cited first, 1 of 1)

- [SUFLECA: Scaling Up Feature Learning for CAD-to-image Alignment](https://mlsanity.com/paper/2607.15058/) (2026)

## Abstract (arXiv)

Foundation features from self-supervised vision models and text-to-image diffusion models have proven effective for semantic correspondence estimation. However, because these features are learned primarily from 2D image objectives, they lack explicit 3D awareness and often confuse symmetric object sides, repeated parts, and visually similar structures that are distinct in 3D. We introduce a 3D-aware post-training framework that goes beyond available 2D foundation features by incorporating priors from 3D foundation models. Given an image, our method uses SAM3D to estimate object geometry and pose, and refines the pose through render-and-compare optimization. Subsequently, we render PartField descriptors from the reconstructed geometry into the image plane based on the estimated object pose. The resulting geometry-aware feature maps complement DINO and Stable Diffusion features, while geodesic distances on the reconstructed shapes enable reliable filtering of candidate correspondences. We use the filtered matches as supervision to train a lightweight adapter on top of DINO and Stable Diffusion for semantic correspondence. In contrast to prior post-training approaches that require pose annotations and rely on coarse spherical geometry, our method automatically obtains instance-specific 3D structure and uses it to guide correspondence learning. Experiments show that our approach improves semantic correspondence over the prior methods while reducing manual geometric supervision. Code and model can be found at https:/github.com/GenIntel/3D-SC.