# Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objects arXiv: https://arxiv.org/abs/2401.00405 ยท HTML: https://mlsanity.com/paper/2401.00405/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 47 references that are in the map. - First posted on arXiv 31 Dec 2023. ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017) - [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015) - [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021) - [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018) - [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020) - [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018) - [3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics](https://mlsanity.com/paper/2011.09127/) (2020) - [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020) - [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019) - [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018) - [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018) - [IM2CAD](https://mlsanity.com/paper/1608.05137/) (2016) - [Triplet-Center Loss for Multi-View 3D Object Retrieval](https://mlsanity.com/paper/1803.06189/) (2018) - [ROCA: Robust CAD Model Retrieval and Alignment from a Single Image](https://mlsanity.com/paper/2112.01988/) (2021) - [Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve](https://mlsanity.com/paper/2007.13034/) (2020) - [Templates for 3D Object Pose Estimation Revisited: Generalization to New Objects and Robustness to Occlusions](https://mlsanity.com/paper/2203.17234/) (2022) - [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021) - [Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image](https://mlsanity.com/paper/2108.09368/) (2021) - [GIFT: A Real-time and Scalable 3D Shape Search Engine](https://mlsanity.com/paper/1604.01879/) (2016) - [From Points to Multi-Object 3D Reconstruction](https://mlsanity.com/paper/2012.11575/) (2020) - [SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB image](https://mlsanity.com/paper/2210.01044/) (2022) - [Location Field Descriptors: Single Image 3D Model Retrieval in the Wild](https://mlsanity.com/paper/1908.02853/) (2019) - [Hard Example Generation by Texture Synthesis for Cross-domain Shape Similarity Learning](https://mlsanity.com/paper/2010.12238/) (2020) - [Deep Exemplar 2D-3D Detection by Adapting from Real to Rendered Views](https://mlsanity.com/paper/1512.02497/) (2015) - [3D Pose Estimation for Fine-Grained Object Categories](https://mlsanity.com/paper/1806.04314/) (2018) ## Built on by (most cited first, 8 of 8) - [Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modeling](https://mlsanity.com/paper/2411.19492/) (2024) - [Towards Scalable and Consistent 3D Editing](https://mlsanity.com/paper/2510.02994/) (2025) - [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025) - [JRM: Joint Reconstruction Model for Multiple Objects without Alignment](https://mlsanity.com/paper/2603.25985/) (2026) - [PReP: Efficient context-based shape retrieval for missing parts](https://mlsanity.com/paper/2410.14245/) (2024) - [Material Magic Wand: Material-Aware Grouping of 3D Parts in Untextured Meshes](https://mlsanity.com/paper/2603.17370/) (2026) - [PASR: Pose-Aware 3D Shape Retrieval from Occluded Single Views](https://mlsanity.com/paper/2604.22658/) (2026) - [DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusion](https://mlsanity.com/paper/2605.16807/) (2026) ## Abstract (arXiv) Single-view 3D shape retrieval is a challenging task that is increasingly important with the growth of available 3D data. Prior work that has studied this task has not focused on evaluating how realistic occlusions impact performance, and how shape retrieval methods generalize to scenarios where either the target 3D shape database contains unseen shapes, or the input image contains unseen objects. In this paper, we systematically evaluate single-view 3D shape retrieval along three different axes: the presence of object occlusions and truncations, generalization to unseen 3D shape data, and generalization to unseen objects in the input images. We standardize two existing datasets of real images and propose a dataset generation pipeline to produce a synthetic dataset of scenes with multiple objects exhibiting realistic occlusions. Our experiments show that training on occlusion-free data as was commonly done in prior work leads to significant performance degradation for inputs with occlusion. We find that that by first pretraining on our synthetic dataset with occlusions and then finetuning on real data, we can significantly outperform models from prior work and demonstrate robustness to both unseen 3D shapes and unseen objects.