mlsanityData to 7 Oct 2026

A Survey on RGB-D Datasets

arXiv 2201.05761 ↗First posted 15 Jan 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

RGB-D data is essential for solving many problems in computer vision. Hundreds of public RGB-D datasets containing various scenes, such as indoor, outdoor, aerial, driving, and medical, have been proposed. These datasets are useful for different applications and are fundamental for addressing classic computer vision tasks, such as monocular depth estimation. This paper reviewed and categorized image datasets that include depth information. We gathered 203 datasets that contain accessible data and grouped them into three categories: scene/objects, body, and medical. We also provided an overview of the different types of sensors, depth applications, and we examined trends and future directions of the usage and creation of datasets containing depth data, and how they can be applied to investigate the development of generalizable machine learning models in the monocular depth estimation field.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 101 of its 265 references that are in the map. First posted on arXiv 15 Jan 2022.

Builds on

101 of 265 references.

References · 101

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  6. 2021Vision Transformers for Dense Predictioncited by 1,789
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  11. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  12. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
38 more
  1. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  2. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
  3. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  4. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  5. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  6. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
  7. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  8. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  9. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  10. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  11. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  12. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  13. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  14. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  15. 2020Virtual KITTI 2cited by 345
  16. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  17. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  18. 2017Sparsity Invariant CNNscited by 325
  19. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 297
  20. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  21. 2018The Multi Vehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perceptioncited by 251
  22. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 239
  23. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  24. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  25. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  26. 2020Consistent Video Depth Estimationcited by 220
  27. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  28. 2016DeMoN: Depth and Motion Network for Learning Monocular Stereocited by 200
  29. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  30. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  31. 2020Robust Consistent Video Depth Estimationcited by 188
  32. 2016Single-Image Depth Perception in the Wildcited by 179
  33. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  34. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  35. 2019WoodScape: A multi-task, multi-camera fisheye dataset for autonomous drivingcited by 142
  36. 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 137
  37. 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
  38. 20193D Ken Burns Effect from a Single Imagecited by 128

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 5; 2026: 1.

11 papers.

  1. 2023Salient Object Detection in RGB-D Videoscited by 8
  2. 2024X-Prompt: Multi-modal Visual Prompt for Video Object Segmentationcited by 7
  3. 2022Outdoor Monocular Depth Estimation: A Research Reviewcited by 4
  4. 2024D4D: An RGBD diffusion model to boost monocular depth estimationcited by 3
  5. 2025Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Taskscited by 2
  6. 2025ColabSfM: Collaborative Structure-from-Motion by Point Cloud Registrationcited by 2
  7. 2023AutoDepthNet: High Frame Rate Depth Map Reconstruction using Commodity Depth and RGB Camerascited by 1
  8. 2025Depth-aware Fusion Method based on Image and 4D Radar Spectrum for 3D Object Detectioncited by 1
  9. 2025Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Taskscited by 1
  10. 2025HiddenObject: Modality-Agnostic Fusion for Multimodal Hidden Object Detectioncited by 0
  11. 2026Filling the Pareto-Optimal Front for Affordance Segmentation on Embedded Devices Using RGB-D Camerascited by 0