# Conjugate Product Graphs for Globally Optimal 2D-3D Shape Matching

arXiv: https://arxiv.org/abs/2211.11589 · HTML: https://mlsanity.com/paper/2211.11589/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 63 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017)
- [Line Graph Neural Networks for Link Prediction](https://mlsanity.com/paper/2010.10046/) (2020)
- [Multilayer Networks](https://mlsanity.com/paper/1309.7233/) (2013)
- [Correspondence Learning via Linearly-invariant Embedding](https://mlsanity.com/paper/2010.13136/) (2020)
- [PoseTrack: Joint Multi-Person Pose Estimation and Tracking](https://mlsanity.com/paper/1611.07727/) (2016)
- [Sketch-based 3D Shape Retrieval using Convolutional Neural Networks](https://mlsanity.com/paper/1504.03504/) (2015)
- [Efficient Decomposition of Image and Mesh Graphs by Lifted Multicuts](https://mlsanity.com/paper/1505.06973/) (2015)
- [DS*: Tighter Lifting-Free Convex Relaxations for Quadratic Matching Problems](https://mlsanity.com/paper/1711.10733/) (2017)
- [A Scalable Combinatorial Solver for Elastic Geometrically Consistent 3D Shape Matching](https://mlsanity.com/paper/2204.12805/) (2022)
- [SHREC'22 Track: Sketch-Based 3D Shape Retrieval in the Wild](https://mlsanity.com/paper/2207.04945/) (2022)
- [A Combinatorial Solution to Non-Rigid 3D Shape-to-Image Matching](https://mlsanity.com/paper/1611.05241/) (2016)

## Built on by (most cited first, 6 of 6)

- [LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching](https://mlsanity.com/paper/2306.11925/) (2023)
- [Geometrically Consistent Partial Shape Matching](https://mlsanity.com/paper/2309.05013/) (2023)
- [DiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape Matching](https://mlsanity.com/paper/2310.08230/) (2023)
- [Fast Globally Optimal and Geometrically Consistent 3D Shape Matching](https://mlsanity.com/paper/2504.06385/) (2025)
- [SIGMA: Scale-Invariant Global Sparse Shape Matching](https://mlsanity.com/paper/2308.08393/) (2023)
- [An Integer Linear Programming Approach to Geometrically Consistent Partial-Partial Shape Matching](https://mlsanity.com/paper/2602.06590/) (2026)

## Abstract (arXiv)

We consider the problem of finding a continuous and non-rigid matching between a 2D contour and a 3D mesh. While such problems can be solved to global optimality by finding a shortest path in the product graph between both shapes, existing solutions heavily rely on unrealistic prior assumptions to avoid degenerate solutions (e.g. knowledge to which region of the 3D shape each point of the 2D contour is matched). To address this, we propose a novel 2D-3D shape matching formalism based on the conjugate product graph between the 2D contour and the 3D shape. Doing so allows us for the first time to consider higher-order costs, i.e. defined for edge chains, as opposed to costs defined for single edges. This offers substantially more flexibility, which we utilise to incorporate a local rigidity prior. By doing so, we effectively circumvent degenerate solutions and thereby obtain smoother and more realistic matchings, even when using only a one-dimensional feature descriptor. Overall, our method finds globally optimal and continuous 2D-3D matchings, has the same asymptotic complexity as previous solutions, produces state-of-the-art results for shape matching and is even capable of matching partial shapes. Our code is publicly available (https://github.com/paul0noah/sm-2D3D).