# Fostering Generalization in Single-view 3D Reconstruction by Learning a Hierarchy of Local and Global Shape Priors

arXiv: https://arxiv.org/abs/2104.00476 · HTML: https://mlsanity.com/paper/2104.00476/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 42 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Superquadrics Revisited: Learning 3D Shape Parsing beyond Cuboids](https://mlsanity.com/paper/1904.09970/) (2019)
- [PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes](https://mlsanity.com/paper/1911.10949/) (2019)
- [3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks](https://mlsanity.com/paper/1708.01648/) (2017)
- [Learning Unsupervised Hierarchical Part Decomposition of 3D Objects from a Single RGB Image](https://mlsanity.com/paper/2004.01176/) (2020)
- [Learning to Reconstruct Shapes from Unseen Classes](https://mlsanity.com/paper/1812.11166/) (2018)
- [Im2Struct: Recovering 3D Shape Structure from a Single RGB Image](https://mlsanity.com/paper/1804.05469/) (2018)
- [Learning Part Generation and Assembly for Structure-aware Shape Synthesis](https://mlsanity.com/paper/1906.06693/) (2019)
- [Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers](https://mlsanity.com/paper/1804.10975/) (2018)

## Built on by (most cited first, 9 of 9)

- [ABO: Dataset and Benchmarks for Real-World 3D Object Understanding](https://mlsanity.com/paper/2110.06199/) (2021)
- [ZeroShape: Regression-based Zero-shot Shape Reconstruction](https://mlsanity.com/paper/2312.14198/) (2023)
- [Topologically-Aware Deformation Fields for Single-View 3D Reconstruction](https://mlsanity.com/paper/2205.06267/) (2022)
- [PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories](https://mlsanity.com/paper/2206.04916/) (2022)
- [Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation](https://mlsanity.com/paper/2403.01619/) (2024)
- [Pose Adaptive Dual Mixup for Few-Shot Single-View 3D Reconstruction](https://mlsanity.com/paper/2112.12484/) (2021)
- [Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes](https://mlsanity.com/paper/2408.14279/) (2024)
- [3D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Data](https://mlsanity.com/paper/2302.12883/) (2023)
- [SRAM: Shape-Realism Alignment Metric for No Reference 3D Shape Evaluation](https://mlsanity.com/paper/2512.01373/) (2025)

## Abstract (arXiv)

Single-view 3D object reconstruction has seen much progress, yet methods still struggle generalizing to novel shapes unseen during training. Common approaches predominantly rely on learned global shape priors and, hence, disregard detailed local observations. In this work, we address this issue by learning a hierarchy of priors at different levels of locality from ground truth input depth maps. We argue that exploiting local priors allows our method to efficiently use input observations, thus improving generalization in visible areas of novel shapes. At the same time, the combination of local and global priors enables meaningful hallucination of unobserved parts resulting in consistent 3D shapes. We show that the hierarchical approach generalizes much better than the global approach. It generalizes not only between different instances of a class but also across classes and to unseen arrangements of objects.