mlsanityData to 7 Oct 2026

Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Image

arXiv 2108.09368 ↗First posted 20 Aug 2021cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

3D perception of object shapes from RGB image input is fundamental towards semantic scene understanding, grounding image-based perception in our spatially 3-dimensional real-world environments. To achieve a mapping between image views of objects and 3D shapes, we leverage CAD model priors from existing large-scale databases, and propose a novel approach towards constructing a joint embedding space between 2D images and 3D CAD models in a patch-wise fashion -- establishing correspondences between patches of an image view of an object and patches of CAD geometry. This enables part similarity reasoning for retrieving similar CADs to a new image view without exact matches in the database. Our patch embedding provides more robust CAD retrieval for shape estimation in our end-to-end estimation of CAD model shape and pose for detected objects in a single input image. Experiments on in-the-wild, complex imagery from ScanNet show that our approach is more robust than state of the art in real-world scenarios without any exact CAD matches.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 40 of its 60 references that are in the map. First posted on arXiv 20 Aug 2021.

Builds on

40 of 60 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2017Mask R-CNNcited by 7,259
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
  7. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  8. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  9. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  10. 2015Fast R-CNNcited by 2,930
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
28 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  2. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  3. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  4. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  5. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  7. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  8. 2019PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flowscited by 487
  9. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  10. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  11. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  12. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  13. 2019CvxNet: Learnable Convex Decompositioncited by 173
  14. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  15. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  16. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103
  17. 2019Rethinking Classification and Localization for Object Detectioncited by 98
  18. 2016IM2CADcited by 78
  19. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  20. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  21. 2018Scan2Mesh: From Unstructured Range Scans to 3D Meshescited by 51
  22. 2020SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scanscited by 44
  23. 2019ShapeMask: Learning to Segment Novel Objects by Refining Shape Priorscited by 42
  24. 2019End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scanscited by 40
  25. 2016Marr Revisited: 2D-3D Alignment via Surface Normal Predictioncited by 34
  26. 2019Location Field Descriptors: Single Image 3D Model Retrieval in the Wildcited by 15
  27. 2015Deep Exemplar 2D-3D Detection by Adapting from Real to Rendered Viewscited by 7
  28. 2018Scene Recomposition by Learning-based ICPcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 7; 2023: 9; 2024: 8; 2025: 5; 2026: 7.

39 papers.

Sort:

By area:

  1. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  2. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  3. 2021ROCA: Robust CAD Model Retrieval and Alignment from a Single Imagecited by 66
  4. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  5. 2023Mesh2Tex: Generating Mesh Textures from Image Queriescited by 31
  6. 2022Learning 3D Scene Priors with 2D Supervisioncited by 24
  7. 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
  8. 2021Pose2Room: Understanding 3D Scenes from Human Activitiescited by 22
  9. 2023BUOL: A Bottom-Up Framework with Occupancy-aware Lifting for Panoptic 3D Scene Reconstruction From A Single Imagecited by 20
  10. 2022SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB imagecited by 19
  11. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 18
  12. 2022RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformerscited by 17
27 more
  1. 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14
  2. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  3. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 9
  4. 2023Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objectscited by 8
  5. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  6. 2022PatchRD: Detail-Preserving Shape Completion by Learning Patch Retrieval and Deformationcited by 7
  7. 2024FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videoscited by 7
  8. 2024Reconstructing Animals and the Wildcited by 7
  9. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  10. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  11. 2022IC3D: Image-Conditioned 3D Diffusion for Shape Generationcited by 5
  12. 2023Architext: Language-Driven Generative Architecture Designcited by 5
  13. 2021Leveraging Geometry for Shape Estimation from a Single RGB Imagecited by 4
  14. 2023MXM-CLR: A Unified Framework for Contrastive Learning of Multifold Cross-Modal Representationscited by 3
  15. 2023Sparse Multi-Object Render-and-Comparecited by 3
  16. 2022Weakly-Supervised End-to-End CAD Retrieval to Scan Objectscited by 2
  17. 2023CA2T-Net: Category-Agnostic 3D Articulation Transfer from Single Imagecited by 2
  18. 2024Can Shape-Infused Joint Embeddings Improve Image-Conditioned 3D Diffusion?cited by 1
  19. 2024PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Imagescited by 1
  20. 2025Zero-shot Inexact CAD Model Alignment from a Single Imagecited by 1
  21. 2026MessyKitchens: Contact-rich object-level 3D scene reconstructioncited by 1
  22. 2022Review on 6D Object Pose Estimation with the focus on Indoor Scene Understandingcited by 0
  23. 2026TIMI: Training-Free Image-to-3D Multi-Instance Generation with Spatial Fidelitycited by 0
  24. 2026DecoRec: Decomposed 3D Scene Reconstruction from Single-View Images via Object-Level Diffusioncited by 0
  25. 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Imagecited by 0
  26. 2026Interpretable Human-Label-Free Deep Learning for Real-Bogus Classification with Uncertainty Quantificationcited by 0
  27. 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generationcited by 0