mlsanityData to 7 Oct 2026

Estimating Generic 3D Room Structures from 2D Annotations

arXiv 2306.09077 ↗First posted 15 Jun 2023cs.CV cs.GRcited by 6

Not on the map. Why?

Abstract (arXiv)

Indoor rooms are among the most common use cases in 3D scene understanding. Current state-of-the-art methods for this task are driven by large annotated datasets. Room layouts are especially important, consisting of structural elements in 3D, such as wall, floor, and ceiling. However, they are difficult to annotate, especially on pure RGB video. We propose a novel method to produce generic 3D room layouts just from 2D segmentation masks, which are easy to annotate for humans. Based on these 2D annotations, we automatically reconstruct 3D plane equations for the structural elements and their spatial extent in the scene, and connect adjacent elements at the appropriate contact edges. We annotate and publicly release 2246 3D room layouts on the RealEstate10k dataset, containing YouTube videos. We demonstrate the high quality of these 3D layouts annotations with extensive experiments.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 45 of its 79 references that are in the map. First posted on arXiv 15 Jun 2023.

Builds on

45 of 79 references.

References · 45

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  8. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  9. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  10. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  11. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  12. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 360
33 more
  1. 2016ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentationcited by 330
  2. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  3. 2018Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomizationcited by 292
  4. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  5. 2019Connecting Vision and Language with Localized Narrativescited by 225
  6. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  7. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  8. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  9. 2017Deep Extreme Cut: From Extreme Points to Object Segmentationcited by 127
  10. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  11. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 107
  12. 2018PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Imagecited by 105
  13. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 94
  14. 2018InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Datasetcited by 87
  15. 2018PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Imagecited by 77
  16. 2021Enhancing Photorealism Enhancementcited by 74
  17. 2020OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasetscited by 73
  18. 2020DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusioncited by 68
  19. 2018DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panoramacited by 67
  20. 2020CoReNet: Coherent 3D scene reconstruction from a single RGB imagecited by 45
  21. 2020SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scanscited by 44
  22. 2017RoomNet: End-to-End Room Layout Estimationcited by 38
  23. 2018Interactive Full Image Segmentation by Considering All Regions Jointlycited by 34
  24. 2020Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videoscited by 33
  25. 2017Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimationcited by 29
  26. 2021PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Cloudscited by 26
  27. 2020General 3D Room Layout from a Single View by Render-and-Comparecited by 21
  28. 2016Pano2CAD: Room Layout From A Single Panorama Imagecited by 20
  29. 2023From Semi-supervised to Omni-supervised Room Layout Estimation Using Point Cloudscited by 18
  30. 2018Guide Me: Interacting with Deep Networkscited by 16
  31. 2016A Coarse-to-Fine Indoor Layout Estimation (CFILE) Methodcited by 15
  32. 2020GeoLayout: Geometry Driven Room Layout Estimation Based on Depth Maps of Planescited by 12
  33. 2022Photo-realistic Neural Domain Randomizationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 4.

6 papers.

  1. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  2. 2025HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generationcited by 22
  3. 2024AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddingscited by 13
  4. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  5. 2025Unposed Sparse Views Room Layout Reconstruction in the Age of Pretrain Modelcited by 3
  6. 2025HouseLayout3D: A Benchmark and Training-Free Baseline for 3D Layout Estimation in the Wildcited by 1