# Localized Shape Modelling with Global Coherence: An Inverse Spectral Approach

arXiv: https://arxiv.org/abs/2108.02161 · HTML: https://mlsanity.com/paper/2108.02161/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 4 Aug 2021.

## Builds on (most cited first, 30 of 32)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [3D Morphable Face Models -- Past, Present and Future](https://mlsanity.com/paper/1909.01815/) (2019)
- [Generating 3D faces using Convolutional Mesh Autoencoders](https://mlsanity.com/paper/1807.10267/) (2018)
- [Learning Gradient Fields for Shape Generation](https://mlsanity.com/paper/2008.06520/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [Combining Implicit Function Learning and Parametric Models for 3D Human Reconstruction](https://mlsanity.com/paper/2007.11432/) (2020)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [3D Shape Segmentation with Projective Convolutional Networks](https://mlsanity.com/paper/1612.02808/) (2016)
- [Unsupervised Shape and Pose Disentanglement for 3D Meshes](https://mlsanity.com/paper/2007.11341/) (2020)
- [StructureNet: Hierarchical Graph Networks for 3D Shape Generation](https://mlsanity.com/paper/1908.00575/) (2019)
- [LIMP: Learning Latent Shape Representations with Metric Preservation Priors](https://mlsanity.com/paper/2003.12283/) (2020)
- [DECOR-GAN: 3D Shape Detailization by Conditional Refinement](https://mlsanity.com/paper/2012.09159/) (2020)
- [PointGMM: a Neural GMM Network for Point Clouds](https://mlsanity.com/paper/2003.13326/) (2020)
- [L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention](https://mlsanity.com/paper/1908.00720/) (2019)
- [Geometric Disentanglement for Generative Latent Shape Models](https://mlsanity.com/paper/1908.06386/) (2019)
- [COALESCE: Component Assembly by Learning to Synthesize Connections](https://mlsanity.com/paper/2008.01936/) (2020)
- [FARM: Functional Automatic Registration Method for 3D Human Bodies](https://mlsanity.com/paper/1807.10517/) (2018)
- [Learning Generative Models of Shape Handles](https://mlsanity.com/paper/2004.03028/) (2020)
- [Disentangled Human Body Embedding Based on Deep Hierarchical Neural Network](https://mlsanity.com/paper/1905.05622/) (2019)
- [DEMEA: Deep Mesh Autoencoders for Non-Rigidly Deforming Objects](https://mlsanity.com/paper/1905.10290/) (2019)
- [Isospectralization, or how to hear shape, style, and correspondence](https://mlsanity.com/paper/1811.11465/) (2018)
- [Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55](https://mlsanity.com/paper/1710.06104/) (2017)
- [Universal Spectral Adversarial Attacks for Deformable Shapes](https://mlsanity.com/paper/2104.03356/) (2021)
- [Localized Manifold Harmonics for Spectral Shape Analysis](https://mlsanity.com/paper/1707.02596/) (2017)
- [Correspondence-Free Region Localization for Partial Shape Similarity via Hamiltonian Spectrum Alignment](https://mlsanity.com/paper/1906.06226/) (2019)
- [Instant recovery of shape from spectrum via latent space connections](https://mlsanity.com/paper/2003.06523/) (2020)

## Abstract (arXiv)

Many natural shapes have most of their characterizing features concentrated over a few regions in space. For example, humans and animals have distinctive head shapes, while inorganic objects like chairs and airplanes are made of well-localized functional parts with specific geometric features. Often, these features are strongly correlated -- a modification of facial traits in a quadruped should induce changes to the body structure. However, in shape modelling applications, these types of edits are among the hardest ones; they require high precision, but also a global awareness of the entire shape. Even in the deep learning era, obtaining manipulable representations that satisfy such requirements is an open problem posing significant constraints. In this work, we address this problem by defining a data-driven model upon a family of linear operators (variants of the mesh Laplacian), whose spectra capture global and local geometric properties of the shape at hand. Modifications to these spectra are translated to semantically valid deformations of the corresponding surface. By explicitly decoupling the global from the local surface features, our pipeline allows to perform local edits while simultaneously maintaining a global stylistic coherence. We empirically demonstrate how our learning-based model generalizes to shape representations not seen at training time, and we systematically analyze different choices of local operators over diverse shape categories.