mlsanityData to 14 Oct 2026

DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image

arXiv 2209.13362 ↗First posted 27 Sep 2022cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

Light-weight time-of-flight (ToF) depth sensors are small, cheap, low-energy and have been massively deployed on mobile devices for the purposes like autofocus, obstacle detection, etc. However, due to their specific measurements (depth distribution in a region instead of the depth value at a certain pixel) and extremely low resolution, they are insufficient for applications requiring high-fidelity depth such as 3D reconstruction. In this paper, we propose DELTAR, a novel method to empower light-weight ToF sensors with the capability of measuring high resolution and accurate depth by cooperating with a color image. As the core of DELTAR, a feature extractor customized for depth distribution and an attention-based neural architecture is proposed to fuse the information from the color and ToF domain efficiently. To evaluate our system in real-world scenarios, we design a data collection device and propose a new approach to calibrate the RGB camera and ToF sensor. Experiments show that our method produces more accurate depth than existing frameworks designed for depth completion and depth super-resolution and achieves on par performance with a commodity-level RGB-D sensor. Code and data are available at https://zju3dv.github.io/deltar/.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 38 of its 61 references that are in the map. First posted on arXiv 27 Sep 2022.

Builds on

38 of 61 references.

References · 38

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2021Vision Transformers for Dense Predictioncited by 1,803
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  10. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
26 more
  1. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  2. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  3. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  4. 2021A Survey of Transformerscited by 401
  5. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  6. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  7. 2020Non-Local Spatial Propagation Network for Depth Completioncited by 195
  8. 2020Binary Neural Networks: A Surveycited by 190
  9. 2021PENet: Towards Precise and Efficient Image Guided Depth Completioncited by 146
  10. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 145
  11. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  12. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  13. 2018Deep Depth Completion of a Single RGB-D Imagecited by 110
  14. 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 110
  15. 2019CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completioncited by 110
  16. 2019Depth Completion from Sparse LiDAR Data with Depth-Normal Constraintscited by 108
  17. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  18. 2017Learning Affinity via Spatial Propagation Networkscited by 79
  19. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  20. 2020Learning Joint 2D-3D Representations for Depth Completioncited by 71
  21. 2017Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimationcited by 70
  22. 2021Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Predictioncited by 48
  23. 2018ActiveStereoNet: End-to-End Self-Supervised Learning for Active Stereo Systemscited by 35
  24. 2018Detail Preserving Depth Estimation from a Single Image Using Attention Guided Networkscited by 31
  25. 2019Generating and Exploiting Probabilistic Monocular Depth Estimatescited by 18
  26. 2018Plug-and-Play: Improve Depth Estimation via Sparse Data Propagationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 5; 2024: 12; 2025: 7; 2026: 1.

26 papers.

Sort:

  1. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  2. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 74
  3. 2024CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Fieldcited by 61
  4. 2023PATS: Patch Area Transportation with Subdivision for Local Feature Matchingcited by 33
  5. 2023BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimationcited by 33
  6. 2023CP-SLAM: Collaborative Neural Point-based SLAM Systemcited by 32
  7. 2023Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensorcited by 30
  8. 2022Consistent Direct Time-of-Flight Video Depth Super-Resolutioncited by 16
  9. 2024Towards 3D Vision with Low-Cost Single-Photon Camerascited by 15
  10. 2024BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Eventscited by 13
  11. 2024Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGBcited by 10
  12. 2024RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methodscited by 8
14 more
  1. 2024ETO:Efficient Transformer-based Local Feature Matching by Organizing Multiple Homography Hypothesescited by 6
  2. 2024BlinkTrack: Feature Tracking over 80 FPS via Events and Imagescited by 4
  3. 2024CFPNet: Improving Lightweight ToF Depth Completion via Cross-zone Feature Propagationcited by 4
  4. 2024ARMOR: Egocentric Perception for Humanoid Robot Collision Avoidance and Motion Planningcited by 4
  5. 2025SVDC: Consistent Direct Time-of-Flight Video Depth Completion with Frequency Selective Fusioncited by 4
  6. 2024Using a Distance Sensor to Detect Deviations in a Planar Surfacecited by 3
  7. 2024A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embeddingcited by 3
  8. 2025DEPTHOR: Depth Enhancement from a Practical Light-Weight dToF Sensor and RGB Imagecited by 3
  9. 2025Recovering Parametric Scenes from Very Few Time-of-Flight Pixelscited by 3
  10. 2025Towards High-Precision Depth Sensing via Monocular-Aided iToF and RGB Integrationcited by 1
  11. 2025DEPTHOR++: Robust Depth Enhancement from a Real-World Lightweight dToF and RGB Guidancecited by 1
  12. 2025ToF-Splatting: Dense SLAM using Sparse Time-of-Flight Depth and Multi-Frame Integrationcited by 0
  13. 2025Hardware-aware Coding Function Design for Compressive Single-Photon 3D Camerascited by 0
  14. 2026Dense Metric Depth Completion from Sparse Direct Time-of-Flight Sensorscited by 0