# Zero-Shot Image Feature Consensus with Deep Functional Maps

arXiv: https://arxiv.org/abs/2403.12038 · HTML: https://mlsanity.com/paper/2403.12038/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 57 references that are in the map.
- First posted on arXiv 18 Mar 2024.

## Builds on (most cited first, 30 of 35)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence](https://mlsanity.com/paper/2305.15347/) (2023)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [SPair-71k: A Large-scale Benchmark for Semantic Correspondence](https://mlsanity.com/paper/1908.10543/) (2019)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [Unsupervised Deep Learning for Structured Shape Matching](https://mlsanity.com/paper/1812.03794/) (2018)
- [CATs: Cost Aggregation Transformers for Visual Correspondence](https://mlsanity.com/paper/2106.02520/) (2021)
- [GAN-Supervised Dense Visual Alignment](https://mlsanity.com/paper/2112.05143/) (2021)
- [Partial Functional Correspondence](https://mlsanity.com/paper/1506.05274/) (2015)
- [End-to-end weakly-supervised semantic alignment](https://mlsanity.com/paper/1712.06861/) (2017)
- [DPFM: Deep Partial Functional Maps](https://mlsanity.com/paper/2110.09994/) (2021)
- [GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network](https://mlsanity.com/paper/2009.07823/) (2020)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [Unsupervised Deep Multi-Shape Matching](https://mlsanity.com/paper/2207.09610/) (2022)
- [Deep Orientation-Aware Functional Maps: Tackling Symmetry Issues in Shape Matching](https://mlsanity.com/paper/2204.13453/) (2022)
- [Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching](https://mlsanity.com/paper/2210.06373/) (2022)
- [Recurrent Transformer Networks for Semantic Correspondence](https://mlsanity.com/paper/1810.12155/) (2018)

## Built on by (most cited first, 10 of 10)

- [Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuning](https://mlsanity.com/paper/2411.19458/) (2024)
- [Do It Yourself: Learning Semantic Correspondence from Pseudo-Labels](https://mlsanity.com/paper/2506.05312/) (2025)
- [Jamais Vu: Exposing the Generalization Gap in Supervised Semantic Correspondence](https://mlsanity.com/paper/2506.08220/) (2025)
- [GECO: Geometrically Consistent Embedding with Lightspeed Inference](https://mlsanity.com/paper/2508.00746/) (2025)
- [ProTracker: Probabilistic Integration for Robust and Accurate Point Tracking](https://mlsanity.com/paper/2501.03220/) (2025)
- [Semantic Correspondence: Unified Benchmarking and a Strong Baseline](https://mlsanity.com/paper/2505.18060/) (2025)
- [Towards Geometry-Grounded Dense Semantic Matching with VGGT Priors](https://mlsanity.com/paper/2509.21263/) (2025)
- [DeepShapeMatchingKit: Accelerated Functional Map Solver and Shape Matching Pipelines Revisited](https://mlsanity.com/paper/2604.10377/) (2026)
- [Quantum-Inspired Harmonic Decision Models: A Computational Framework for Music Generation](https://mlsanity.com/paper/2607.05007/) (2026)
- [MatRes: Zero-Shot Test-Time Model Adaptation for Simultaneous Matching and Restoration](https://mlsanity.com/paper/2604.10081/) (2026)

## Abstract (arXiv)

Correspondences emerge from large-scale vision models trained for generative and discriminative tasks. This has been revealed and benchmarked by computing correspondence maps between pairs of images, using nearest neighbors on the feature grids. Existing work has attempted to improve the quality of these correspondence maps by carefully mixing features from different sources, such as by combining the features of different layers or networks. We point out that a better correspondence strategy is available, which directly imposes structure on the correspondence field: the functional map. Wielding this simple mathematical tool, we lift the correspondence problem from the pixel space to the function space and directly optimize for mappings that are globally coherent. We demonstrate that our technique yields correspondences that are not only smoother but also more accurate, with the possibility of better reflecting the knowledge embedded in the large-scale vision models that we are studying. Our approach sets a new state-of-the-art on various dense correspondence tasks. We also demonstrate our effectiveness in keypoint correspondence and affordance map transfer.