# Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds

arXiv: https://arxiv.org/abs/2106.03437 · HTML: https://mlsanity.com/paper/2106.03437/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 7 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [SDM-NET: Deep Generative Network for Structured Deformable Mesh](https://mlsanity.com/paper/1908.04520/) (2019)
- [BAE-NET: Branched Autoencoder for Shape Co-Segmentation](https://mlsanity.com/paper/1903.11228/) (2019)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [Im2Struct: Recovering 3D Shape Structure from a Single RGB Image](https://mlsanity.com/paper/1804.05469/) (2018)
- [PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation](https://mlsanity.com/paper/1903.00709/) (2019)
- [Learning Part Generation and Assembly for Structure-aware Shape Synthesis](https://mlsanity.com/paper/1906.06693/) (2019)
- [Composite Shape Modeling via Latent Space Factorization](https://mlsanity.com/paper/1901.02968/) (2019)
- [Deep Parametric Shape Predictions using Distance Fields](https://mlsanity.com/paper/1904.08921/) (2019)
- [Learning Implicit Functions for Topology-Varying Dense 3D Shape Correspondence](https://mlsanity.com/paper/2010.12320/) (2020)
- [Learning Generative Models of Shape Handles](https://mlsanity.com/paper/2004.03028/) (2020)
- [GRASS: Generative Recursive Autoencoders for Shape Structures](https://mlsanity.com/paper/1705.02090/) (2017)
- [SAGNet:Structure-aware Generative Network for 3D-Shape Modeling](https://mlsanity.com/paper/1808.03981/) (2018)
- [Neural Star Domain as Primitive Representation](https://mlsanity.com/paper/2010.11248/) (2020)
- [SEG-MAT: 3D Shape Segmentation Using Medial Axis Transform](https://mlsanity.com/paper/2010.11488/) (2020)

## Built on by (most cited first, 30 of 44)

- [GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts](https://mlsanity.com/paper/2211.05272/) (2022)
- [SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model](https://mlsanity.com/paper/2403.13064/) (2024)
- [SALAD: Part-Level Latent Diffusion for 3D Shape Generation and Manipulation](https://mlsanity.com/paper/2303.12236/) (2023)
- [D2CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts](https://mlsanity.com/paper/2301.11497/) (2023)
- [PartAfford: Part-level Affordance Discovery from 3D Objects](https://mlsanity.com/paper/2202.13519/) (2022)
- [Self-Supervised Category-Level Articulated Object Pose Estimation with Part-Level SE(3) Equivariance](https://mlsanity.com/paper/2302.14268/) (2023)
- [ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives](https://mlsanity.com/paper/2305.05661/) (2023)
- [EditVAE: Unsupervised Part-Aware Controllable 3D Point Cloud Shape Generation](https://mlsanity.com/paper/2110.06679/) (2021)
- [SuperDec: 3D Scene Decomposition with Superquadric Primitives](https://mlsanity.com/paper/2504.00992/) (2025)
- [RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures](https://mlsanity.com/paper/2201.12763/) (2022)
- [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023)
- [Primitive-based Shape Abstraction via Nonparametric Bayesian Inference](https://mlsanity.com/paper/2203.14714/) (2022)
- [CasaGPT: Cuboid Arrangement and Scene Assembly for Interior Design](https://mlsanity.com/paper/2504.19478/) (2025)
- [DPF-Net: Combining Explicit Shape Priors in Deformable Primitive Field for Unsupervised Structural Reconstruction of 3D Objects](https://mlsanity.com/paper/2308.13225/) (2023)
- [PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization](https://mlsanity.com/paper/2502.12985/) (2025)
- [Unsupervised 3D Shape Reconstruction by Part Retrieval and Assembly](https://mlsanity.com/paper/2303.01999/) (2023)
- [DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation](https://mlsanity.com/paper/2311.13125/) (2023)
- [Spice-E : Structural Priors in 3D Diffusion using Cross-Entity Attention](https://mlsanity.com/paper/2311.17834/) (2023)
- [PriFit: Learning to Fit Primitives Improves Few Shot Point Cloud Segmentation](https://mlsanity.com/paper/2112.13942/) (2021)
- [ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points](https://mlsanity.com/paper/2503.02745/) (2025)
- [Light-SQ: Structure-aware Shape Abstraction with Superquadrics for Generated Meshes](https://mlsanity.com/paper/2509.24986/) (2025)
- [SweepNet: Unsupervised Learning Shape Abstraction via Neural Sweepers](https://mlsanity.com/paper/2407.06305/) (2024)
- [CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Image](https://mlsanity.com/paper/2504.11230/) (2025)
- [Residual Primitive Fitting of 3D Shapes with SuperFrusta](https://mlsanity.com/paper/2512.09201/) (2025)
- [HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling](https://mlsanity.com/paper/2301.10460/) (2023)
- [A Survey of Methods for Converting Unstructured Data to CSG Models](https://mlsanity.com/paper/2305.01220/) (2023)
- [GenAnalysis: Joint Shape Analysis by Learning Man-Made Shape Generators with Deformation Regularizations](https://mlsanity.com/paper/2503.00807/) (2025)
- [You Only Estimate Once: Unified, One-stage, Real-Time Category-level Articulated Object 6D Pose Estimation for Robotic Grasping](https://mlsanity.com/paper/2506.05719/) (2025)
- [SuperFlex: Deformable Superquadrics for Point Cloud Decomposition](https://mlsanity.com/paper/2607.01015/) (2026)
- [SPA-VAE: Similar-Parts-Assignment for Unsupervised 3D Point Cloud Generation](https://mlsanity.com/paper/2203.07825/) (2022)

## Abstract (arXiv)

Representing complex 3D objects as simple geometric primitives, known as shape abstraction, is important for geometric modeling, structural analysis, and shape synthesis. In this paper, we propose an unsupervised shape abstraction method to map a point cloud into a compact cuboid representation. We jointly predict cuboid allocation as part segmentation and cuboid shapes and enforce the consistency between the segmentation and shape abstraction for self-learning. For the cuboid abstraction task, we transform the input point cloud into a set of parametric cuboids using a variational auto-encoder network. The segmentation network allocates each point into a cuboid considering the point-cuboid affinity. Without manual annotations of parts in point clouds, we design four novel losses to jointly supervise the two branches in terms of geometric similarity and cuboid compactness. We evaluate our method on multiple shape collections and demonstrate its superiority over existing shape abstraction methods. Moreover, based on our network architecture and learned representations, our approach supports various applications including structured shape generation, shape interpolation, and structural shape clustering.