# Im2Struct: Recovering 3D Shape Structure from a Single RGB Image

arXiv: https://arxiv.org/abs/1804.05469 · HTML: https://mlsanity.com/paper/1804.05469/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2018.

## Built on by (most cited first, 30 of 55)

- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects](https://mlsanity.com/paper/2410.16499/) (2024)
- [Deep Implicit Moving Least-Squares Functions for 3D Reconstruction](https://mlsanity.com/paper/2103.12266/) (2021)
- [CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assembly](https://mlsanity.com/paper/2104.05652/) (2021)
- [Part123: Part-aware 3D Reconstruction from a Single-view Image](https://mlsanity.com/paper/2405.16888/) (2024)
- [Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds](https://mlsanity.com/paper/2106.03437/) (2021)
- [D2CSG: Unsupervised Learning of Compact CSG Trees with Dual Complements and Dropouts](https://mlsanity.com/paper/2301.11497/) (2023)
- [SceneHGN: Hierarchical Graph Networks for 3D Indoor Scene Generation with Fine-Grained Geometry](https://mlsanity.com/paper/2302.10237/) (2023)
- [PartNeRF: Generating Part-Aware Editable 3D Shapes without 3D Supervision](https://mlsanity.com/paper/2303.09554/) (2023)
- [Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes](https://mlsanity.com/paper/2206.04942/) (2022)
- [Robust and Accurate Superquadric Recovery: a Probabilistic Approach](https://mlsanity.com/paper/2111.14517/) (2021)
- [Learning Compositional Representation for 4D Captures with Neural ODE](https://mlsanity.com/paper/2103.08271/) (2021)
- [Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images](https://mlsanity.com/paper/2204.09866/) (2022)
- [RIM-Net: Recursive Implicit Fields for Unsupervised Learning of Hierarchical Shape Structures](https://mlsanity.com/paper/2201.12763/) (2022)
- [Marching-Primitives: Shape Abstraction from Signed Distance Function](https://mlsanity.com/paper/2303.13190/) (2023)
- [Cuboids Revisited: Learning Robust 3D Shape Fitting to Single RGB Images](https://mlsanity.com/paper/2105.02047/) (2021)
- [ANISE: Assembly-based Neural Implicit Surface rEconstruction](https://mlsanity.com/paper/2205.13682/) (2022)
- [Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization](https://mlsanity.com/paper/2305.03043/) (2023)
- [Primitive-based Shape Abstraction via Nonparametric Bayesian Inference](https://mlsanity.com/paper/2203.14714/) (2022)
- [Translating a Visual LEGO Manual to a Machine-Executable Plan](https://mlsanity.com/paper/2207.12572/) (2022)
- [PASTA: Controllable Part-Aware Shape Generation with Autoregressive Transformers](https://mlsanity.com/paper/2407.13677/) (2024)
- [Hierarchical Motion Understanding via Motion Programs](https://mlsanity.com/paper/2104.11216/) (2021)
- [CAD-Estate: Large-scale CAD Model Annotation in RGB Videos](https://mlsanity.com/paper/2306.09011/) (2023)
- [Unsupervised Pose-Aware Part Decomposition for 3D Articulated Objects](https://mlsanity.com/paper/2110.04411/) (2021)
- [DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Image](https://mlsanity.com/paper/2309.12594/) (2023)
- [DPF-Net: Combining Explicit Shape Priors in Deformable Primitive Field for Unsupervised Structural Reconstruction of 3D Objects](https://mlsanity.com/paper/2308.13225/) (2023)
- [Fostering Generalization in Single-view 3D Reconstruction by Learning a Hierarchy of Local and Global Shape Priors](https://mlsanity.com/paper/2104.00476/) (2021)
- [Test-time Adaptation with Slot-Centric Models](https://mlsanity.com/paper/2203.11194/) (2022)
- [PartSDF: Part-Based Implicit Neural Representation for Composite 3D Shape Parametrization and Optimization](https://mlsanity.com/paper/2502.12985/) (2025)
- [Self-supervised Learning of Hybrid Part-aware 3D Representations of 2D Gaussians and Superquadrics](https://mlsanity.com/paper/2408.10789/) (2024)

## Abstract (arXiv)

We propose to recover 3D shape structures from single RGB images, where structure refers to shape parts represented by cuboids and part relations encompassing connectivity and symmetry. Given a single 2D image with an object depicted, our goal is automatically recover a cuboid structure of the object parts as well as their mutual relations. We develop a convolutional-recursive auto-encoder comprised of structure parsing of a 2D image followed by structure recovering of a cuboid hierarchy. The encoder is achieved by a multi-scale convolutional network trained with the task of shape contour estimation, thereby learning to discern object structures in various forms and scales. The decoder fuses the features of the structure parsing network and the original image, and recursively decodes a hierarchy of cuboids. Since the decoder network is learned to recover part relations including connectivity and symmetry explicitly, the plausibility and generality of part structure recovery can be ensured. The two networks are jointly trained using the training data of contour-mask and cuboid structure pairs. Such pairs are generated by rendering stock 3D CAD models coming with part segmentation. Our method achieves unprecedentedly faithful and detailed recovery of diverse 3D part structures from single-view 2D images. We demonstrate two applications of our method including structure-guided completion of 3D volumes reconstructed from single-view images and structure-aware interactive editing of 2D images.