mlsanityData to 14 Oct 2026

ROCA: Robust CAD Model Retrieval and Alignment from a Single Image

arXiv 2112.01988 ↗First posted 3 Dec 2021cs.CV cs.GR cs.LGcited by 67

Not on the map. Why?

Abstract (arXiv)

We present ROCA, a novel end-to-end approach that retrieves and aligns 3D CAD models from a shape database to a single input image. This enables 3D perception of an observed scene from a 2D RGB observation, characterized as a lightweight, compact, clean CAD representation. Core to our approach is our differentiable alignment optimization based on dense 2D-3D object correspondences and Procrustes alignment. ROCA can thus provide a robust CAD alignment while simultaneously informing CAD retrieval by leveraging the 2D-3D correspondences to learn geometrically similar CAD models. Experiments on challenging, real-world imagery from ScanNet show that ROCA significantly improves on state of the art, from 9.5% to 17.6% in retrieval-aware CAD alignment accuracy.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

42 of 54 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2017Mask R-CNNcited by 7,277
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  9. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  10. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,434
  11. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  12. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
30 more
  1. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  2. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  3. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  4. 2017Focal Loss for Dense Object Detectioncited by 752
  5. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  6. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  7. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  8. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  9. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  10. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  11. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  12. 2020Deep Global Registrationcited by 229
  13. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  14. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 172
  15. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 145
  16. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  17. 2016IM2CADcited by 78
  18. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  19. 2019End-to-End Learnable Geometric Vision by Backpropagating PnP Optimizationcited by 65
  20. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  21. 2018Scan2Mesh: From Unstructured Range Scans to 3D Meshescited by 51
  22. 2019ShapeMask: Learning to Segment Novel Objects by Refining Shape Priorscited by 43
  23. 2019End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scanscited by 40
  24. 2021Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Imagecited by 39
  25. 2020Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videoscited by 33
  26. 2020From Points to Multi-Object 3D Reconstructioncited by 29
  27. 20193D-RelNet: Joint Object and Relational Network for 3D Predictioncited by 28
  28. 2021Voxel-based 3D Detection and Reconstruction of Multiple Objects from a Single Imagecited by 20
  29. 2019Joint Embedding of 3D Scan and CAD Objectscited by 19
  30. 2020CAD-Deform: Deformable Fitting of CAD Models to 3D Scanscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 10; 2024: 14; 2025: 14; 2026: 22.

67 papers.

Sort:

By area:

  1. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 117
  2. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 106
  3. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 84
  4. 2025SceneGen: Single-Image 3D Scene Generation in One Feedforward Passcited by 54
  5. 2023DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Imagecited by 48
  6. 2023ShapeGPT: 3D Shape Generation with A Unified Multi-modal Language Modelcited by 39
  7. 2024GenCAD: Image-Conditioned Computer-Aided Design Generation with Transformer-Based Contrastive Representation and Diffusion Priorscited by 38
  8. 2024SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphscited by 29
  9. 2024Line-based 6-DoF Object Pose Estimation and Tracking With an Event Cameracited by 29
  10. 2025CADDreamer: CAD Object Generation from Single-view Imagescited by 25
  11. 2022Learning 3D Scene Priors with 2D Supervisioncited by 24
  12. 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
38 more
  1. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  2. 2025LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scanscited by 21
  3. 2023Visual Localization using Imperfect 3D Models from the Internetcited by 20
  4. 2022SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB imagecited by 19
  5. 2025WorldGen: From Text to Traversable and Interactive 3D Worldscited by 19
  6. 2022RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformerscited by 17
  7. 2022ObjectMatch: Robust Registration using Canonical Object Correspondencescited by 15
  8. 2022Certifiable 3D Object Pose Estimation: Foundations, Learning Models, and Self-Trainingcited by 12
  9. 2023U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Cloudscited by 12
  10. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 11
  11. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 11
  12. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  13. 2023ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformationcited by 8
  14. 2023Generalizing Single-View 3D Shape Retrieval to Occlusions and Unseen Objectscited by 8
  15. 2024Coherent 3D Scene Diffusion From a Single RGB Imagecited by 8
  16. 2024Learning Instance-Aware Correspondences for Robust Multi-Instance Point Cloud Registration in Cluttered Scenescited by 7
  17. 2024Reconstructing Animals and the Wildcited by 7
  18. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  19. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
  20. 2022CAD 3D Model classification by Graph Neural Networks: A new approach based on STEP formatcited by 6
  21. 2024KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformationcited by 6
  22. 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Datasetcited by 5
  23. 2025Learning 3D Object Spatial Relationships from Pre-trained 2D Diffusion Modelscited by 5
  24. 2023CA2T-Net: Category-Agnostic 3D Articulation Transfer from Single Imagecited by 3
  25. 2023SketchANIMAR: Sketch-based 3D Animal Fine-Grained Retrievalcited by 3
  26. 2023Sparse Multi-Object Render-and-Comparecited by 3
  27. 2024WALT3D: Generating Realistic Training Data from Time-Lapse Imagery for Reconstructing Dynamic Objects under Occlusioncited by 3
  28. 2025MADrive: Memory-Augmented Driving Scene Modelingcited by 3
  29. 2025TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulationcited by 3
  30. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  31. 2022Weakly-Supervised End-to-End CAD Retrieval to Scan Objectscited by 2
  32. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  33. 2026OSCAR: Open-Set CAD Retrieval from a Language Prompt and a Single Imagecited by 2
  34. 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignmentcited by 2
  35. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  36. 2026WorldSculpt: Generating Compositional Worlds from Grounded Videoscited by 2
  37. 2024fCOP: Focal Length Estimation from Category-level Object Priorscited by 1
  38. 2025Zero-shot Inexact CAD Model Alignment from a Single Imagecited by 1

50 most cited of 67 All 67 (JSON)