# Shape My Face: Registering 3D Face Scans by Surface-to-Surface Translation

arXiv: https://arxiv.org/abs/2012.09235 · HTML: https://mlsanity.com/paper/2012.09235/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 14 of 14)

- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [Instant Multi-View Head Capture through Learnable Registration](https://mlsanity.com/paper/2306.07437/) (2023)
- [ScanTalk: 3D Talking Heads from Unregistered Scans](https://mlsanity.com/paper/2403.10942/) (2024)
- [3DMM-RF: Convolutional Radiance Fields for 3D Face Modeling](https://mlsanity.com/paper/2209.07366/) (2022)
- [ImFace++: A Sophisticated Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2312.04028/) (2023)
- [Unsupervised Diffeomorphic Surface Registration and Non-Linear Modelling](https://mlsanity.com/paper/2109.13630/) (2021)
- [Toward Mesh-Invariant 3D Generative Deep Learning with Geometric Measures](https://mlsanity.com/paper/2306.15762/) (2023)
- [Learning Implicit Functions for Dense 3D Shape Correspondence of Generic Objects](https://mlsanity.com/paper/2212.14276/) (2022)
- [Automatic 3D Registration of Dental CBCT and Face Scan Data using 2D Projection Images](https://mlsanity.com/paper/2305.10132/) (2023)
- [3D Face Modeling via Weakly-supervised Disentanglement Network joint Identity-consistency Prior](https://mlsanity.com/paper/2404.16536/) (2024)
- [Feed-forward Gaussian Registration for Head Avatar Creation and Editing](https://mlsanity.com/paper/2603.15811/) (2026)
- [Representing 3D Faces with Learnable B-Spline Volumes](https://mlsanity.com/paper/2604.12894/) (2026)
- [Registration-Free Learnable Multi-View Capture of Faces in Dense Semantic Correspondence](https://mlsanity.com/paper/2605.01450/) (2026)
- [FreeTalk: Emotional Topology-Free 3D Talking Heads](https://mlsanity.com/paper/2603.15512/) (2026)

## Abstract (arXiv)

Standard registration algorithms need to be independently applied to each surface to register, following careful pre-processing and hand-tuning. Recently, learning-based approaches have emerged that reduce the registration of new scans to running inference with a previously-trained model. In this paper, we cast the registration task as a surface-to-surface translation problem, and design a model to reliably capture the latent geometric information directly from raw 3D face scans. We introduce Shape-My-Face (SMF), a powerful encoder-decoder architecture based on an improved point cloud encoder, a novel visual attention mechanism, graph convolutional decoders with skip connections, and a specialized mouth model that we smoothly integrate with the mesh convolutions. Compared to the previous state-of-the-art learning algorithms for non-rigid registration of face scans, SMF only requires the raw data to be rigidly aligned (with scaling) with a pre-defined face template. Additionally, our model provides topologically-sound meshes with minimal supervision, offers faster training time, has orders of magnitude fewer trainable parameters, is more robust to noise, and can generalize to previously unseen datasets. We extensively evaluate the quality of our registrations on diverse data. We demonstrate the robustness and generalizability of our model with in-the-wild face scans across different modalities, sensor types, and resolutions. Finally, we show that, by learning to register scans, SMF produces a hybrid linear and non-linear morphable model. Manipulation of the latent space of SMF allows for shape generation, and morphing applications such as expression transfer in-the-wild. We train SMF on a dataset of human faces comprising 9 large-scale databases on commodity hardware.