# Generating 3D faces using Convolutional Mesh Autoencoders

arXiv: https://arxiv.org/abs/1807.10267 · HTML: https://mlsanity.com/paper/1807.10267/

## Key facts

- Cited by 268 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jul 2018.

## Built on by (most cited first, 30 of 268)

- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [I M Avatar: Implicit Morphable Head Avatars from Videos](https://mlsanity.com/paper/2112.07471/) (2021)
- [SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networks](https://mlsanity.com/paper/2104.03313/) (2021)
- [Reconstructing Personalized Semantic Facial NeRF Models From Monocular Video](https://mlsanity.com/paper/2210.06108/) (2022)
- [Pixel Codec Avatars](https://mlsanity.com/paper/2104.04638/) (2021)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding](https://mlsanity.com/paper/2312.02214/) (2023)
- [Deep Generative Models in Engineering Design: A Review](https://mlsanity.com/paper/2110.10863/) (2021)
- [Graph Stacked Hourglass Networks for 3D Human Pose Estimation](https://mlsanity.com/paper/2103.16385/) (2021)
- [Interacting Attention Graph for Single Image Two-Hand Reconstruction](https://mlsanity.com/paper/2203.09364/) (2022)
- [Compositional Human-Scene Interaction Synthesis with Semantic Control](https://mlsanity.com/paper/2207.12824/) (2022)
- [Driving-Signal Aware Full-Body Avatars](https://mlsanity.com/paper/2105.10441/) (2021)
- [Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers](https://mlsanity.com/paper/2207.13820/) (2022)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [CameraHMR: Aligning People with Perspective](https://mlsanity.com/paper/2411.08128/) (2024)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [Subdivision-Based Mesh Convolution Networks](https://mlsanity.com/paper/2106.02285/) (2021)
- [A Survey of Non-Rigid 3D Registration](https://mlsanity.com/paper/2203.07858/) (2022)
- [Shape registration in the time of transformers](https://mlsanity.com/paper/2106.13679/) (2021)
- [LEMON: Learning 3D Human-Object Interaction Relation from 2D Images](https://mlsanity.com/paper/2312.08963/) (2023)
- [GIFS: Neural Implicit Function for General Shape Representation](https://mlsanity.com/paper/2204.07126/) (2022)
- [A graph convolutional autoencoder approach to model order reduction for parametrized PDEs](https://mlsanity.com/paper/2305.08573/) (2023)
- [Learning Disentangled Avatars with Hybrid 3D Representations](https://mlsanity.com/paper/2309.06441/) (2023)
- [URAvatar: Universal Relightable Gaussian Codec Avatars](https://mlsanity.com/paper/2410.24223/) (2024)
- [Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image](https://mlsanity.com/paper/2208.13241/) (2022)
- [Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey](https://mlsanity.com/paper/2411.03688/) (2024)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [Normalized Avatar Synthesis Using StyleGAN and Perceptual Refinement](https://mlsanity.com/paper/2106.11423/) (2021)

## Abstract (arXiv)

Learned 3D representations of human faces are useful for computer vision problems such as 3D face tracking and reconstruction from images, as well as graphics applications such as character generation and animation. Traditional models learn a latent representation of a face using linear subspaces or higher-order tensor generalizations. Due to this linearity, they can not capture extreme deformations and non-linear expressions. To address this, we introduce a versatile model that learns a non-linear representation of a face using spectral convolutions on a mesh surface. We introduce mesh sampling operations that enable a hierarchical mesh representation that captures non-linear variations in shape and expression at multiple scales within the model. In a variational setting, our model samples diverse realistic 3D faces from a multivariate Gaussian distribution. Our training data consists of 20,466 meshes of extreme expressions captured over 12 different subjects. Despite limited training data, our trained model outperforms state-of-the-art face models with 50% lower reconstruction error, while using 75% fewer parameters. We also show that, replacing the expression space of an existing state-of-the-art face model with our autoencoder, achieves a lower reconstruction error. Our data, model and code are available at http://github.com/anuragranj/coma