# Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching

arXiv: https://arxiv.org/abs/2303.10971 · HTML: https://mlsanity.com/paper/2303.10971/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 63 references that are in the map.
- First posted on arXiv 20 Mar 2023.

## Builds on (most cited first, 30 of 30)

- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [DiffusionNet: Discretization Agnostic Learning on Surfaces](https://mlsanity.com/paper/2012.00888/) (2020)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Deep Functional Maps: Structured Prediction for Dense Shape Correspondence](https://mlsanity.com/paper/1704.08686/) (2017)
- [Deep Geometric Functional Maps: Robust Feature Learning for Shape Correspondence](https://mlsanity.com/paper/2003.14286/) (2020)
- [Unsupervised Deep Learning for Structured Shape Matching](https://mlsanity.com/paper/1812.03794/) (2018)
- [NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go](https://mlsanity.com/paper/2106.09431/) (2021)
- [Continuous and Orientation-preserving Correspondences via Functional Maps](https://mlsanity.com/paper/1806.04455/) (2018)
- [Deep Shells: Unsupervised Shape Correspondence with Optimal Transport](https://mlsanity.com/paper/2010.15261/) (2020)
- [Smooth Shells: Multi-Scale Shape Registration with Functional Maps](https://mlsanity.com/paper/1905.12512/) (2019)
- [Partial Functional Correspondence](https://mlsanity.com/paper/1506.05274/) (2015)
- [DPFM: Deep Partial Functional Maps](https://mlsanity.com/paper/2110.09994/) (2021)
- [Correspondence Learning via Linearly-invariant Embedding](https://mlsanity.com/paper/2010.13136/) (2020)
- [CorrNet3D: Unsupervised End-to-end Learning of Dense Correspondence for 3D Point Clouds](https://mlsanity.com/paper/2012.15638/) (2020)
- [Unsupervised Deep Multi-Shape Matching](https://mlsanity.com/paper/2207.09610/) (2022)
- [Deep Orientation-Aware Functional Maps: Tackling Symmetry Issues in Shape Matching](https://mlsanity.com/paper/2204.13453/) (2022)
- [Dual Geometric Graph Network (DG2N) -- Iterative network for deformable shape alignment](https://mlsanity.com/paper/2011.14723/) (2020)
- [Isometric Multi-Shape Matching](https://mlsanity.com/paper/2012.02689/) (2020)
- [Product Manifold Filter: Non-Rigid Shape Correspondence via Kernel Density Estimation in the Product Space](https://mlsanity.com/paper/1701.00669/) (2017)
- [Structured Regularization of Functional Map Computations](https://mlsanity.com/paper/2009.14624/) (2020)
- [Bending Graphs: Hierarchical Shape Matching using Gated Optimal Transport](https://mlsanity.com/paper/2202.01537/) (2022)
- [Unsupervised cycle-consistent deformation for shape matching](https://mlsanity.com/paper/1907.03165/) (2019)
- [MINA: Convex Mixed-Integer Programming for Non-Rigid Shape Alignment](https://mlsanity.com/paper/2002.12623/) (2020)
- [A Scalable Combinatorial Solver for Elastic Geometrically Consistent 3D Shape Matching](https://mlsanity.com/paper/2204.12805/) (2022)
- [Implicit field supervision for robust non-rigid shape matching](https://mlsanity.com/paper/2203.07694/) (2022)

## Built on by (most cited first, 23 of 23)

- [LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matching](https://mlsanity.com/paper/2306.11925/) (2023)
- [Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features](https://mlsanity.com/paper/2311.17024/) (2023)
- [Zero-Shot 3D Shape Correspondence](https://mlsanity.com/paper/2306.03253/) (2023)
- [Spectral Meets Spatial: Harmonising 3D Shape Matching and Interpolation](https://mlsanity.com/paper/2402.18920/) (2024)
- [Hybrid Functional Maps for Crease-Aware Non-Isometric Shape Matching](https://mlsanity.com/paper/2312.03678/) (2023)
- [Denoising Functional Maps: Diffusion Models for Shape Correspondence](https://mlsanity.com/paper/2503.01845/) (2025)
- [Hierarchical Neural Semantic Representation for 3D Semantic Correspondence](https://mlsanity.com/paper/2509.17431/) (2025)
- [Integrating Efficient Optimal Transport and Functional Maps For Unsupervised Shape Correspondence Learning](https://mlsanity.com/paper/2403.01781/) (2024)
- [Revisiting Map Relations for Unsupervised Non-Rigid Shape Matching](https://mlsanity.com/paper/2310.11420/) (2023)
- [DiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape Matching](https://mlsanity.com/paper/2310.08230/) (2023)
- [Unsupervised Template-assisted Point Cloud Shape Correspondence Network](https://mlsanity.com/paper/2403.16412/) (2024)
- [Synchronous Diffusion for Unsupervised Smooth Non-Rigid 3D Shape Matching](https://mlsanity.com/paper/2407.08244/) (2024)
- [ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Models](https://mlsanity.com/paper/2410.02615/) (2024)
- [Kissing to Find a Match: Efficient Low-Rank Permutation Representation](https://mlsanity.com/paper/2308.13252/) (2023)
- [Occlusion-aware Non-Rigid Point Cloud Registration via Unsupervised Neural Deformation Correntropy](https://mlsanity.com/paper/2502.10704/) (2025)
- [MorphAny3D: Unleashing the Power of Structured Latent in 3D Morphing](https://mlsanity.com/paper/2601.00204/) (2026)
- [Equivariant Local Reference Frames for Unsupervised Non-rigid Point Cloud Shape Correspondence](https://mlsanity.com/paper/2404.00959/) (2024)
- [PAOLI: Pose-free Articulated Object Learning from Sparse-view Images](https://mlsanity.com/paper/2509.04276/) (2025)
- [GLASS: Graph and Vision-Language Assisted Semantic Shape Correspondence](https://mlsanity.com/paper/2603.07652/) (2026)
- [DeepShapeMatchingKit: Accelerated Functional Map Solver and Shape Matching Pipelines Revisited](https://mlsanity.com/paper/2604.10377/) (2026)
- [Hyper-Network Neural Functional Maps for Unsupervised Robust 3D Shape Matching](https://mlsanity.com/paper/2606.30131/) (2026)
- [Matching Shapes Under Different Topologies: A Topology-Adaptive Deformation Guided Approach](https://mlsanity.com/paper/2509.06862/) (2025)
- [TokenMatch: 3D Mesh Correspondence Transformer with Curvature-Guided Tokenisation](https://mlsanity.com/paper/2609.04202/) (2026)

## Abstract (arXiv)

The matching of 3D shapes has been extensively studied for shapes represented as surface meshes, as well as for shapes represented as point clouds. While point clouds are a common representation of raw real-world 3D data (e.g. from laser scanners), meshes encode rich and expressive topological information, but their creation typically requires some form of (often manual) curation. In turn, methods that purely rely on point clouds are unable to meet the matching quality of mesh-based methods that utilise the additional topological structure. In this work we close this gap by introducing a self-supervised multimodal learning strategy that combines mesh-based functional map regularisation with a contrastive loss that couples mesh and point cloud data. Our shape matching approach allows to obtain intramodal correspondences for triangle meshes, complete point clouds, and partially observed point clouds, as well as correspondences across these data modalities. We demonstrate that our method achieves state-of-the-art results on several challenging benchmark datasets even in comparison to recent supervised methods, and that our method reaches previously unseen cross-dataset generalisation ability.