# Learning Generative Models of Shape Handles

arXiv: https://arxiv.org/abs/2004.03028 · HTML: https://mlsanity.com/paper/2004.03028/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 25 of 25)

- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects](https://mlsanity.com/paper/2410.16499/) (2024)
- [Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Search](https://mlsanity.com/paper/2205.02961/) (2022)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds](https://mlsanity.com/paper/2106.03437/) (2021)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds](https://mlsanity.com/paper/2109.00113/) (2021)
- [Category-Level Multi-Part Multi-Joint 3D Shape Assembly](https://mlsanity.com/paper/2303.06163/) (2023)
- [GLASS: Geometric Latent Augmentation for Shape Spaces](https://mlsanity.com/paper/2108.03225/) (2021)
- [ANISE: Assembly-based Neural Implicit Surface rEconstruction](https://mlsanity.com/paper/2205.13682/) (2022)
- [PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers](https://mlsanity.com/paper/2407.13677/) (2024)
- [PrimitiveAnything: Human-Crafted 3D Primitive Assembly Generation with Auto-Regressive Transformer](https://mlsanity.com/paper/2505.04622/) (2025)
- [SPAFormer: Sequential 3D Part Assembly with Transformers](https://mlsanity.com/paper/2403.05874/) (2024)
- [ArtiLatent: Realistic Articulated 3D Object Generation via Structured Latents](https://mlsanity.com/paper/2510.21432/) (2025)
- [3inGAN: Learning a 3D Generative Model from Images of a Self-similar Scene](https://mlsanity.com/paper/2211.14902/) (2022)
- [LSD-StructureNet: Modeling Levels of Structural Detail in 3D Part Hierarchies](https://mlsanity.com/paper/2108.13459/) (2021)
- [Intuitive Shape Editing in Latent Space](https://mlsanity.com/paper/2111.12488/) (2021)
- [Structure-Aware 3D VR Sketch to 3D Shape Retrieval](https://mlsanity.com/paper/2209.09043/) (2022)
- [Convex Decomposition of Indoor Scenes](https://mlsanity.com/paper/2307.04246/) (2023)
- [PriFit: Learning to Fit Primitives Improves Few Shot Point Cloud Segmentation](https://mlsanity.com/paper/2112.13942/) (2021)
- [HybridSDF: Combining Deep Implicit Shapes and Geometric Primitives for 3D Shape Representation and Manipulation](https://mlsanity.com/paper/2109.10767/) (2021)
- [TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly](https://mlsanity.com/paper/2407.15648/) (2024)
- [Improved Convex Decomposition with Ensembling and Negative Primitives](https://mlsanity.com/paper/2405.19569/) (2024)
- [Localized Shape Modelling with Global Coherence: An Inverse Spectral Approach](https://mlsanity.com/paper/2108.02161/) (2021)

## Abstract (arXiv)

We present a generative model to synthesize 3D shapes as sets of handles -- lightweight proxies that approximate the original 3D shape -- for applications in interactive editing, shape parsing, and building compact 3D representations. Our model can generate handle sets with varying cardinality and different types of handles (Figure 1). Key to our approach is a deep architecture that predicts both the parameters and existence of shape handles, and a novel similarity measure that can easily accommodate different types of handles, such as cuboids or sphere-meshes. We leverage the recent advances in semantic 3D annotation as well as automatic shape summarizing techniques to supervise our approach. We show that the resulting shape representations are intuitive and achieve superior quality than previous state-of-the-art. Finally, we demonstrate how our method can be used in applications such as interactive shape editing, completion, and interpolation, leveraging the latent space learned by our model to guide these tasks. Project page: http://mgadelha.me/shapehandles.