mlsanityData to 7 Oct 2026

Automatically Annotating Indoor Images with CAD Models via RGB-D Scans

arXiv 2212.11796 ↗First posted 22 Dec 2022cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

We present an automatic method for annotating images of indoor scenes with the CAD models of the objects by relying on RGB-D scans. Through a visual evaluation by 3D experts, we show that our method retrieves annotations that are at least as accurate as manual annotations, and can thus be used as ground truth without the burden of manually annotating 3D data. We do this using an analysis-by-synthesis approach, which compares renderings of the CAD models with the captured scene. We introduce a 'cloning procedure' that identifies objects that have the same geometry, to annotate these objects with the same CAD models. This allows us to obtain complete annotations for the ScanNet dataset and the recent ARKitScenes dataset.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 15 of its 28 references that are in the map. First posted on arXiv 22 Dec 2022.

Builds on

15 of 28 references.

References · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  6. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  7. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 442
  8. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  9. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  10. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  11. 2018Holistic 3D Scene Parsing and Reconstruction from a Single RGB Imagecited by 69
  12. 2019End-to-End CAD Model Retrieval and 9DoF Alignment in 3D Scanscited by 40
3 more
  1. 20183D Pose Estimation and 3D Model Retrieval for Objects in the Wildcited by 31
  2. 2019Joint Embedding of 3D Scan and CAD Objectscited by 19
  3. 2021Monte Carlo Scene Search for 3D Scene Understandingcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 4; 2026: 1.

9 papers.

  1. 2025LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scanscited by 21
  2. 2024FastCAD: Real-Time CAD Retrieval and Alignment from Scans and Videoscited by 7
  3. 2024PyTorchGeoNodes: Enabling Differentiable Shape Programs for 3D Shape Reconstructioncited by 5
  4. 2023HOC-Search: Efficient CAD Model and Pose Retrieval from RGB-D Scanscited by 3
  5. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  6. 2025SGAligner++: Cross-Modal Language-Aided 3D Scene Graph Alignmentcited by 2
  7. 2025GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfercited by 2
  8. 2024FAWN: Floor-And-Walls Normal Regularization for Direct Neural TSDF Reconstructioncited by 0
  9. 2026CAOA -- Completion-Assisted Object-CAD Alignmentcited by 0