mlsanityData to 14 Oct 2026

R3DS: Reality-linked 3D Scenes for Panoramic Scene Understanding

arXiv 2403.12301 ↗First posted 18 Mar 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce the Reality-linked 3D Scenes (R3DS) dataset of synthetic 3D scenes mirroring the real-world scene arrangements from Matterport3D panoramas. Compared to prior work, R3DS has more complete and densely populated scenes with objects linked to real-world observations in panoramas. R3DS also provides an object support hierarchy, and matching object sets (e.g., same chairs around a dining table) for each scene. Overall, R3DS contains 19K objects represented by 3,784 distinct CAD models from over 100 object categories. We demonstrate the effectiveness of R3DS on the Panoramic Scene Understanding task. We find that: 1) training on R3DS enables better generalization; 2) support relation prediction trained with R3DS improves performance compared to heuristically calculated support; and 3) R3DS offers a challenging benchmark for future work on panoramic scene understanding.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

21 of 31 references.

References · 21

  1. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  4. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  6. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  7. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
  8. 2021ABO: Dataset and Benchmarks for Real-World 3D Object Understandingcited by 366
  9. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 311
  10. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
  11. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  12. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 200
9 more
  1. 2020Rearrangement: A Challenge for Embodied AIcited by 155
  2. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 155
  3. 2018Scan2CAD: Learning CAD Model Alignment in RGB-D Scanscited by 142
  4. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  5. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  6. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 108
  7. 2021DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimizationcited by 33
  8. 2023PanoContext-Former: Panoramic Total Scene Understanding with a Transformercited by 19
  9. 2023CAD-Estate: Large-scale CAD Model Annotation in RGB Videoscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025MetaScenes: Towards Automated Replica Creation for Real-world 3D Scanscited by 31
  2. 2025Are Multimodal Large Language Models Ready for Omnidirectional Spatial Reasoning?cited by 23
  3. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 11
  4. 20263D-Fixer: Coarse-to-Fine In-place Completion for 3D Scenes from a Single Imagecited by 11