mlsanityData to 14 Oct 2026

CAD-Estate: Large-scale CAD Model Annotation in RGB Videos

arXiv 2306.09011 ↗First posted 15 Jun 2023cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

We propose a method for annotating videos of complex multi-object scenes with a globally-consistent 3D representation of the objects. We annotate each object with a CAD model from a database, and place it in the 3D coordinate frame of the scene with a 9-DoF pose transformation. Our method is semi-automatic and works on commonly-available RGB videos, without requiring a depth sensor. Many steps are performed automatically, and the tasks performed by humans are simple, well-specified, and require only limited reasoning in 3D. This makes them feasible for crowd-sourcing and has allowed us to construct a large-scale dataset by annotating real-estate videos from YouTube. Our dataset CAD-Estate offers 101k instances of 12k unique CAD models placed in the 3D representations of 20k videos. In comparison to Scan2CAD, the largest existing dataset with CAD model annotations on real scenes, CAD-Estate has 7x more instances and 4x more unique CAD models. We showcase the benefits of pre-training a Mask2CAD model on CAD-Estate for the task of automatic 3D object reconstruction and pose estimation, demonstrating that it leads to performance improvements on the popular Scan2CAD benchmark. The dataset is available at https://github.com/google-research/cad-estate.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

42 of 54 references.

References · 42

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  5. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  8. 2016Playing for Data: Ground Truth from Computer Gamescited by 818
  9. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 652
  10. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  11. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  12. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
30 more
  1. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  2. 2019Tracking without bells and whistlescited by 382
  3. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  4. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  5. 2018ABC: A Big CAD Model Dataset For Geometric Deep Learningcited by 341
  6. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  7. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 294
  8. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  9. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  10. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  11. 2021ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detectioncited by 157
  12. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  13. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  14. 2018InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Datasetcited by 87
  15. 2017Extreme clicking for efficient object annotationcited by 81
  16. 2021Enhancing Photorealism Enhancementcited by 74
  17. 2020Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrievecited by 65
  18. 2018Im2Struct: Recovering 3D Shape Structure from a Single RGB Imagecited by 55
  19. 2020FroDO: From Detections to 3D Objectscited by 51
  20. 2019SpineNet: Learning Scale-Permuted Backbone for Recognition and Localizationcited by 50
  21. 2020CoReNet: Coherent 3D scene reconstruction from a single RGB imagecited by 45
  22. 2021Patch2CAD: Patchwise Embedding Learning for In-the-Wild Shape Retrieval from a Single Imagecited by 39
  23. 2018Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layerscited by 33
  24. 2020Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videoscited by 33
  25. 2018Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptationcited by 27
  26. 2021ODAM: Object Detection, Association, and Mapping using Posed RGB Videocited by 22
  27. 2020MOLTR: Multiple Object Localisation, Tracking, and Reconstruction from Monocular RGB Videoscited by 18
  28. 2022RayTran: 3D pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformerscited by 17
  29. 2019Graph-RISE: Graph-Regularized Image Semantic Embeddingcited by 15
  30. 2022Photo-realistic Neural Domain Randomizationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 4; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  2. 2024OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objectscited by 22
  3. 2025LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scanscited by 21
  4. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 11
  5. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 11
  6. 2024FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenescited by 10
  7. 2026Engine-Native Editable 3D World Reconstruction with Objects and Lightingcited by 6
  8. 2023LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Datasetcited by 5
  9. 2024R3DS: Reality-linked 3D Scenes for Panoramic Scene Understandingcited by 4
  10. 2025Leveraging Automatic CAD Annotations for Supervised Learning in 3D Scene Understandingcited by 2
  11. 2026WorldSculpt: Generating Compositional Worlds from Grounded Videoscited by 2
  12. 2026UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decompositioncited by 1
2 more
  1. 2025ACD: Direct Conditional Control for Video Diffusion Models via Attention Supervisioncited by 0
  2. 2026SUFLECA: Scaling Up Feature Learning for CAD-to-image Alignmentcited by 0