mlsanityData to 13 Oct 2026

RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methods

arXiv 2405.10357 ↗First posted 16 May 2024cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Integrating an RGB camera into a ToF imaging system has become a significant technique for perceiving the real world. The RGB guided ToF imaging system is crucial to several applications, including face anti-spoofing, saliency detection, and trajectory prediction. Depending on the distance of the working range, the implementation schemes of the RGB guided ToF imaging systems are different. Specifically, ToF sensors with a uniform field of illumination, which can output dense depth but have low resolution, are typically used for close-range measurements. In contrast, LiDARs, which emit laser pulses and can only capture sparse depth, are usually employed for long-range detection. In the two cases, depth quality improvement for RGB guided ToF imaging corresponds to two sub-tasks: guided depth super-resolution and guided depth completion. In light of the recent significant boost to the field provided by deep learning, this paper comprehensively reviews the works related to RGB guided ToF imaging, including network structures, learning strategies, evaluation metrics, benchmark datasets, and objective functions. Besides, we present quantitative comparisons of state-of-the-art methods on widely used benchmark datasets. Finally, we discuss future trends and the challenges in real applications for further research.

Key facts. Cited by 8 papers in the map (data through 13 Oct 2026). Builds on 115 of its 211 references that are in the map. First posted on arXiv 16 May 2024.

Builds on

115 of 211 references.

References · 115

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,656
  2. 2016Densely Connected Convolutional Networkscited by 6,707
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,231
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,440
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,100
  7. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 879
  8. 2017A Survey on Multi-Task Learningcited by 840
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  10. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 779
  11. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 669
  12. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 571
38 more
  1. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  2. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  3. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  4. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  5. 2019Dynamic Convolution: Attention over Convolution Kernelscited by 356
  6. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 352
  7. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 341
  8. 2017Sparsity Invariant CNNscited by 325
  9. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  10. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  11. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 194
  12. 2017Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Imagecited by 159
  13. 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
  14. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  15. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  16. 2019Pixel-Adaptive Convolutional Neural Networkscited by 126
  17. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  18. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  19. 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
  20. 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 109
  21. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  22. 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92
  23. 2019Unsupervised Depth Completion from Visual Inertial Odometrycited by 88
  24. 2019Handheld Multi-Frame Super-Resolutioncited by 85
  25. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  26. 2021RigNet: Repetitive Image Guided Network for Depth Completioncited by 80
  27. 2017Learning Affinity via Spatial Propagation Networkscited by 79
  28. 2019Sparse and noisy LiDAR completion with RGB guidance and uncertaintycited by 79
  29. 2019Deep Convolutional Neural Network for Multi-modal Image Restoration and Fusioncited by 78
  30. 2018Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentationcited by 76
  31. 2021Discrete Cosine Transform Network for Guided Depth Map Super-Resolutioncited by 71
  32. 2022Dynamic Spatial Propagation Network for Depth Completioncited by 70
  33. 2020Adaptive Context-Aware Multi-Modal Network for Depth Completioncited by 69
  34. 2020CASIA-SURF CeFA: A Benchmark for Multi-modal Cross-ethnicity Face Anti-spoofingcited by 66
  35. 2021Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baselinecited by 61
  36. 2021Unsupervised Depth Completion with Calibrated Backprojection Layerscited by 61
  37. 2019Deformable Kernel Networks for Joint Image Filteringcited by 59
  38. 2019Dense Depth Posterior (DDP) from Single Image and Sparse Rangecited by 57

50 most cited of 115 All 115 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2024: 4; 2025: 1; 2026: 3.

8 papers.

  1. 2024CFPNet: Improving Lightweight ToF Depth Completion via Cross-zone Feature Propagationcited by 4
  2. 2024Depth on Demand: Streaming Dense Depth from a Low Frame Rate Active Sensorcited by 2
  3. 2024Analyzing mixed construction and demolition waste in material recovery facilities: evolution, challenges, and applications of computer vision and deep learningcited by 1
  4. 2025Towards High-Precision Depth Sensing via Monocular-Aided iToF and RGB Integrationcited by 1
  5. 2024Active Stereo in the Wild through Virtual Pattern Projectioncited by 0
  6. 2026Real-Time LiDAR Point Cloud Densification for Low-Latency Spatial Data Transmissioncited by 0
  7. 2026ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Cameracited by 0
  8. 2026Reliable iToF Depth Sensing via Sensor-Intrinsic Uncertainty Modeling and State-Space Restorationcited by 0