# Learning Feature Aggregation for Deep 3D Morphable Models

arXiv: https://arxiv.org/abs/2105.02173 · HTML: https://mlsanity.com/paper/2105.02173/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 61 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering](https://mlsanity.com/paper/1606.09375/) (2016)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [Hierarchical Graph Representation Learning with Differentiable Pooling](https://mlsanity.com/paper/1806.08804/) (2018)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Deep Convolutional Networks on Graph-Structured Data](https://mlsanity.com/paper/1506.05163/) (2015)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction](https://mlsanity.com/paper/1703.10580/) (2017)
- [Adaptive Graph Convolutional Neural Networks](https://mlsanity.com/paper/1801.03226/) (2018)
- [HOPE-Net: A Graph-based Model for Hand-Object Pose Estimation](https://mlsanity.com/paper/2004.00060/) (2020)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation](https://mlsanity.com/paper/1905.02876/) (2019)
- [SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator](https://mlsanity.com/paper/1911.05856/) (2019)
- [Variational Autoencoders for Deforming 3D Mesh Models](https://mlsanity.com/paper/1709.04307/) (2017)
- [Fully Convolutional Mesh Autoencoder using Efficient Spatially Varying Kernels](https://mlsanity.com/paper/2006.04325/) (2020)
- [Deformable Shape Completion with Graph Convolutional Autoencoders](https://mlsanity.com/paper/1712.00268/) (2017)
- [Combining 3D Morphable Models: A Large scale Face-and-Head Model](https://mlsanity.com/paper/1903.03785/) (2019)
- [Neural Subdivision](https://mlsanity.com/paper/2005.01819/) (2020)
- [CNNs on Surfaces using Rotation-Equivariant Features](https://mlsanity.com/paper/2006.01570/) (2020)
- [DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes](https://mlsanity.com/paper/2004.01002/) (2020)
- [A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshes](https://mlsanity.com/paper/1809.06664/) (2018)
- [DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects](https://mlsanity.com/paper/1905.10290/) (2019)
- [Spectral Clustering with Graph Neural Networks for Graph Pooling](https://mlsanity.com/paper/1907.00481/) (2019)
- [3D Dense Face Alignment via Graph Convolution Networks](https://mlsanity.com/paper/1904.05562/) (2019)

## Built on by (most cited first, 13 of 13)

- [Learning Neural Parametric Head Models](https://mlsanity.com/paper/2212.02761/) (2022)
- [ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2203.14510/) (2022)
- [Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance](https://mlsanity.com/paper/2401.15687/) (2024)
- [Deep Structural Causal Shape Models](https://mlsanity.com/paper/2208.10950/) (2022)
- [AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging](https://mlsanity.com/paper/2211.07818/) (2022)
- [ImFace++: A Sophisticated Nonlinear 3D Morphable Face Model with Implicit Neural Representations](https://mlsanity.com/paper/2312.04028/) (2023)
- [Anatomically Constrained Implicit Face Models](https://mlsanity.com/paper/2312.07538/) (2023)
- [Tag-based annotation creates better avatars](https://mlsanity.com/paper/2302.07354/) (2023)
- [FaceCom: Towards High-fidelity 3D Facial Shape Completion via Optimization and Inpainting Guidance](https://mlsanity.com/paper/2406.02074/) (2024)
- [Spherical Geometry Diffusion: Generating High-quality 3D Face Geometry via Sphere-anchored Representations](https://mlsanity.com/paper/2601.13371/) (2026)
- [Deep Spectral Meshes: Multi-Frequency Facial Mesh Processing with Graph Neural Networks](https://mlsanity.com/paper/2402.10365/) (2024)
- [3D-GANTex: 3D Face Reconstruction with StyleGAN3-based Multi-View Images and 3DDFA based Mesh Generation](https://mlsanity.com/paper/2410.16009/) (2024)
- [Disentangled Human Body Representation Based on Unsupervised Semantic-Aware Learning](https://mlsanity.com/paper/2505.19049/) (2025)

## Abstract (arXiv)

3D morphable models are widely used for the shape representation of an object class in computer vision and graphics applications. In this work, we focus on deep 3D morphable models that directly apply deep learning on 3D mesh data with a hierarchical structure to capture information at multiple scales. While great efforts have been made to design the convolution operator, how to best aggregate vertex features across hierarchical levels deserves further attention. In contrast to resorting to mesh decimation, we propose an attention based module to learn mapping matrices for better feature aggregation across hierarchical levels. Specifically, the mapping matrices are generated by a compatibility function of the keys and queries. The keys and queries are trainable variables, learned by optimizing the target objective, and shared by all data samples of the same object class. Our proposed module can be used as a train-only drop-in replacement for the feature aggregation in existing architectures for both downsampling and upsampling. Our experiments show that through the end-to-end training of the mapping matrices, we achieve state-of-the-art results on a variety of 3D shape datasets in comparison to existing morphable models.