# Composite Shape Modeling via Latent Space Factorization

arXiv: https://arxiv.org/abs/1901.02968 · HTML: https://mlsanity.com/paper/1901.02968/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2019.

## Built on by (most cited first, 30 of 31)

- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds](https://mlsanity.com/paper/2106.03437/) (2021)
- [D2CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts](https://mlsanity.com/paper/2301.11497/) (2023)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [RGL-NET: A Recurrent Graph Learning framework for Progressive Part Assembly](https://mlsanity.com/paper/2107.12859/) (2021)
- [PartGlot: Learning Shape Part Segmentation from Language Reference Games](https://mlsanity.com/paper/2112.06390/) (2021)
- [Jigsaw: Learning to Assemble Multiple Fractured Objects](https://mlsanity.com/paper/2305.17975/) (2023)
- [Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning](https://mlsanity.com/paper/2405.08054/) (2024)
- [SP-GAN: Sphere-Guided 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2108.04476/) (2021)
- [EditVAE: Unsupervised Part-Aware Controllable 3D Point Cloud Shape Generation](https://mlsanity.com/paper/2110.06679/) (2021)
- [Learning disentangled representations via product manifold projection](https://mlsanity.com/paper/2103.01638/) (2021)
- [3D Part Assembly Generation with Instance Encoded Transformer](https://mlsanity.com/paper/2207.01779/) (2022)
- [ANISE: Assembly-based Neural Implicit Surface rEconstruction](https://mlsanity.com/paper/2205.13682/) (2022)
- [PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers](https://mlsanity.com/paper/2407.13677/) (2024)
- [Score-PA: Score-based 3D Part Assembly](https://mlsanity.com/paper/2309.04220/) (2023)
- [GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis](https://mlsanity.com/paper/2402.16994/) (2024)
- [FLOAT: Factorized Learning of Object Attributes for Improved Multi-object Multi-part Scene Parsing](https://mlsanity.com/paper/2203.16168/) (2022)
- [LSD-StructureNet: Modeling Levels of Structural Detail in 3D Part Hierarchies](https://mlsanity.com/paper/2108.13459/) (2021)
- [Jigsaw++: Imagining Complete Shape Priors for Object Reassembly](https://mlsanity.com/paper/2410.11816/) (2024)
- [A Generative Shape Compositional Framework to Synthesise Populations of Virtual Chimaeras](https://mlsanity.com/paper/2210.01607/) (2022)
- [Point Cloud Part Editing: Segmentation, Generation, Assembly, and Selection](https://mlsanity.com/paper/2312.11867/) (2023)
- [Attention-based Part Assembly for 3D Volumetric Shape Modeling](https://mlsanity.com/paper/2304.10986/) (2023)
- [Self-Supervised Delineation of Geological Structures using Orthogonal Latent Space Projection](https://mlsanity.com/paper/2108.09605/) (2021)
- [SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generation](https://mlsanity.com/paper/2203.07825/) (2022)
- [Quartet of Diffusions: Structure-Aware Point Cloud Generation through Part and Symmetry Guidance](https://mlsanity.com/paper/2601.20425/) (2026)
- [Synthesizing Human Faces using Latent Space Factorization and Local Weights (Extended Version)](https://mlsanity.com/paper/2107.08737/) (2021)
- [3DLatNav: Navigating Generative Latent Spaces for Semantic-Aware 3D Object Manipulation](https://mlsanity.com/paper/2211.09770/) (2022)
- [Deformable Voxel Grids for Shape Comparisons](https://mlsanity.com/paper/2211.11609/) (2022)
- [Diverse Part Synthesis for 3D Shape Creation](https://mlsanity.com/paper/2401.09384/) (2024)

## Abstract (arXiv)

We present a novel neural network architecture, termed Decomposer-Composer, for semantic structure-aware 3D shape modeling. Our method utilizes an auto-encoder-based pipeline, and produces a novel factorized shape embedding space, where the semantic structure of the shape collection translates into a data-dependent sub-space factorization, and where shape composition and decomposition become simple linear operations on the embedding coordinates. We further propose to model shape assembly using an explicit learned part deformation module, which utilizes a 3D spatial transformer network to perform an in-network volumetric grid deformation, and which allows us to train the whole system end-to-end. The resulting network allows us to perform part-level shape manipulation, unattainable by existing approaches. Our extensive ablation study, comparison to baseline methods and qualitative analysis demonstrate the improved performance of the proposed method.