mlsanityData to 7 Oct 2026

ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data

arXiv 2111.08897 ↗First posted 17 Nov 2021cs.CV cs.AIcited by 442

Not on the map. Why?

Cited by 75 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Scene understanding is an active research area. Commercial depth sensors, such as Kinect, have enabled the release of several RGB-D datasets over the past few years which spawned novel methods in 3D scene understanding. More recently with the launch of the LiDAR sensor in Apple's iPads and iPhones, high quality RGB-D data is accessible to millions of people on a device they commonly use. This opens a whole new era in scene understanding for the Computer Vision community as well as app developers. The fundamental research in scene understanding together with the advances in machine learning can now impact people's everyday experiences. However, transforming these scene understanding methods to real-world experiences requires additional innovation and development. In this paper we introduce ARKitScenes. It is not only the first RGB-D dataset that is captured with a now widely available depth sensor, but to our best knowledge, it also is the largest indoor scene understanding data released. In addition to the raw and processed data from the mobile device, ARKitScenes includes high resolution depth maps captured using a stationary laser scanner, as well as manually labeled 3D oriented bounding boxes for a large taxonomy of furniture. We further analyze the usefulness of the data for two downstream tasks: 3D object detection and color-guided depth upsampling. We demonstrate that our dataset can help push the boundaries of existing state-of-the-art methods and it introduces new challenges that better represent real-world scenarios.

Key facts. Cited by 442 papers in the map (data through 7 Oct 2026). Builds on 25 of its 47 references that are in the map. First posted on arXiv 17 Nov 2021.

Builds on

25 of 47 references.

References · 25

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  8. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  9. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  10. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  11. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  12. 2020Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloudcited by 335
13 more
  1. 20183D-SIS: 3D Semantic Instance Segmentation of RGB-D Scanscited by 216
  2. 2021Group-Free 3D Object Detection via Transformerscited by 215
  3. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 168
  4. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  5. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 145
  6. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  7. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  8. 2020HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detectioncited by 82
  9. 2020MLCVNet: Multi-Level Context VoteNet for 3D Object Detectioncited by 80
  10. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  11. 2020FroDO: From Detections to 3D Objectscited by 51
  12. 2020SVGA-Net: Sparse Voxel-Graph Attention Network for 3D Object Detection from Point Cloudscited by 27
  13. 2020Multi-Scale Progressive Fusion Learning for Depth Map Super-Resolutioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 16; 2023: 38; 2024: 60; 2025: 153; 2026: 175.

442 papers.

Sort:

By area:

  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  3. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  4. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  5. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  6. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  7. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  8. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
  9. 20243D Reconstruction with Spatial Memorycited by 269
  10. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  11. 2025AnySplat: Feed-forward 3D Gaussian Splatting from Unconstrained Viewscited by 234
  12. 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 206
38 more
  1. 2025FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Viewscited by 176
  2. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  3. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  4. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  5. 2022Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wildcited by 159
  6. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 154
  7. 2025Streaming 4D Visual Geometry Transformercited by 147
  8. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 139
  9. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  10. 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 133
  11. 2024SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understandingcited by 130
  12. 2024Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generationcited by 129
  13. 2023CompGS: Smaller and Faster Gaussian Splatting with Vector Quantizationcited by 127
  14. 2025From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3Dcited by 127
  15. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  16. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 116
  17. 2023Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understandingcited by 111
  18. 2022Habitat-Matterport 3D Semantics Datasetcited by 109
  19. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 108
  20. 2025GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Modelscited by 106
  21. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  22. 2023Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Trainingcited by 85
  23. 2025Visual Spatial Tuningcited by 85
  24. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  25. 2025Point3R: Streaming 3D Reconstruction with Explicit Spatial Pointer Memorycited by 82
  26. 2024Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localizationcited by 81
  27. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73
  28. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 72
  29. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 71
  30. 2025Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priorscited by 69
  31. 2024OpenNeRF: Open Set 3D Neural Scene Segmentation with Pixel-Wise Features and Rendered Novel Viewscited by 68
  32. 2022Removing Objects From Neural Radiance Fieldscited by 66
  33. 2023Segment3D: Learning Fine-Grained Class-Agnostic 3D Segmentation without Manual Labelscited by 64
  34. 2025SpatialLM: Training Large Language Models for Structured Indoor Modelingcited by 63
  35. 2025Scaling Language-Free Visual Representation Learningcited by 62
  36. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  37. 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
  38. 2025OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modelingcited by 59

50 most cited of 442 All 442 (JSON)