# Locally Adaptive Neural 3D Morphable Models

arXiv: https://arxiv.org/abs/2401.02937 · HTML: https://mlsanity.com/paper/2401.02937/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 46 references that are in the map.
- First posted on arXiv 5 Jan 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion](https://mlsanity.com/paper/2212.06135/) (2022)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction](https://mlsanity.com/paper/1902.05978/) (2019)
- [Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation](https://mlsanity.com/paper/1905.02876/) (2019)
- [SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator](https://mlsanity.com/paper/1911.05856/) (2019)
- [Towards a complete 3D morphable model of the human head](https://mlsanity.com/paper/1911.08008/) (2019)
- [ViTs for SITS: Vision Transformers for Satellite Image Time Series](https://mlsanity.com/paper/2301.04944/) (2023)
- [Combining 3D Morphable Models: A Large scale Face-and-Head Model](https://mlsanity.com/paper/1903.03785/) (2019)
- [3D Face Morphable Models "In-the-Wild"](https://mlsanity.com/paper/1701.05360/) (2017)
- [Persistent Nature: A Generative Model of Unbounded 3D Worlds](https://mlsanity.com/paper/2303.13515/) (2023)
- [A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes](https://mlsanity.com/paper/1809.06664/) (2018)
- [FitMe: Deep Photorealistic 3D Morphable Model Avatars](https://mlsanity.com/paper/2305.09641/) (2023)
- [Relightify: Relightable 3D Faces from a Single Image via Diffusion Models](https://mlsanity.com/paper/2305.06077/) (2023)
- [Geometric Disentanglement for Generative Latent Shape Models](https://mlsanity.com/paper/1908.06386/) (2019)
- [Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild](https://mlsanity.com/paper/2305.08296/) (2023)
- [3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces](https://mlsanity.com/paper/2111.12448/) (2021)
- [3D Generative Model Latent Disentanglement via Local Eigenprojection](https://mlsanity.com/paper/2302.12798/) (2023)
- [3D human tongue reconstruction from single "in-the-wild" images](https://mlsanity.com/paper/2106.12302/) (2021)

## Built on by (most cited first, 5 of 5)

- [WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild](https://mlsanity.com/paper/2409.12259/) (2024)
- [ShapeFusion: A 3D diffusion model for localized shape editing](https://mlsanity.com/paper/2403.19773/) (2024)
- [FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategy](https://mlsanity.com/paper/2503.17197/) (2025)
- [ImHead: A Large-scale Implicit Morphable Model for Localized Head Modeling](https://mlsanity.com/paper/2510.10793/) (2025)
- [Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing](https://mlsanity.com/paper/2511.12529/) (2025)

## Abstract (arXiv)

We present the Locally Adaptive Morphable Model (LAMM), a highly flexible Auto-Encoder (AE) framework for learning to generate and manipulate 3D meshes. We train our architecture following a simple self-supervised training scheme in which input displacements over a set of sparse control vertices are used to overwrite the encoded geometry in order to transform one training sample into another. During inference, our model produces a dense output that adheres locally to the specified sparse geometry while maintaining the overall appearance of the encoded object. This approach results in state-of-the-art performance in both disentangling manipulated geometry and 3D mesh reconstruction. To the best of our knowledge LAMM is the first end-to-end framework that enables direct local control of 3D vertex geometry in a single forward pass. A very efficient computational graph allows our network to train with only a fraction of the memory required by previous methods and run faster during inference, generating 12k vertex meshes at >60fps on a single CPU thread. We further leverage local geometry control as a primitive for higher level editing operations and present a set of derivative capabilities such as swapping and sampling object parts. Code and pretrained models can be found at https://github.com/michaeltrs/LAMM.