# Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments

arXiv: https://arxiv.org/abs/2112.03750 · HTML: https://mlsanity.com/paper/2112.03750/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Burst Denoising with Kernel Prediction Networks](https://mlsanity.com/paper/1712.02327/) (2017)
- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [Learning Guided Convolutional Network for Depth Completion](https://mlsanity.com/paper/1908.01238/) (2019)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Adaptive Context-Aware Multi-Modal Network for Depth Completion](https://mlsanity.com/paper/2008.10833/) (2020)
- [SteReFo: Efficient Image Refocusing with Stereo Vision](https://mlsanity.com/paper/1909.13395/) (2019)
- [Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset](https://mlsanity.com/paper/1807.10376/) (2018)
- [Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module](https://mlsanity.com/paper/1909.07623/) (2019)
- [DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging](https://mlsanity.com/paper/1805.09305/) (2018)
- [Learning to Remove Multipath Distortions in Time-of-Flight Range Images for a Robotic Arm Setup](https://mlsanity.com/paper/1601.01750/) (2016)
- [Applying Depth-Sensing to Automated Surgical Manipulation with a da Vinci Robot](https://mlsanity.com/paper/2002.06302/) (2020)
- [Low Power Depth Estimation of Rigid Objects for Time-of-Flight Imaging](https://mlsanity.com/paper/1810.01930/) (2018)

## Built on by (most cited first, 11 of 11)

- [On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks](https://mlsanity.com/paper/2303.14840/) (2023)
- [MonoGraspNet: 6-DoF Grasping with a Single RGB Image](https://mlsanity.com/paper/2209.13036/) (2022)
- [Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regression](https://mlsanity.com/paper/2205.04565/) (2022)
- [Know your sensORs -- A Modality Study For Surgical Action Classification](https://mlsanity.com/paper/2203.08674/) (2022)
- [Robust Depth Enhancement via Polarization Prompt Fusion Tuning](https://mlsanity.com/paper/2404.04318/) (2024)
- [FloatingFusion: Depth from ToF and Image-stabilized Stereo Cameras](https://mlsanity.com/paper/2210.02785/) (2022)
- [3D Adversarial Augmentations for Robust Out-of-Domain Predictions](https://mlsanity.com/paper/2308.15479/) (2023)
- [S2P3: Self-Supervised Polarimetric Pose Prediction](https://mlsanity.com/paper/2312.01105/) (2023)
- [Multi-Modal Dataset Acquisition for Photometrically Challenging Object](https://mlsanity.com/paper/2308.10621/) (2023)
- [Learnable Burst-Encodable Time-of-Flight Imaging for High-Fidelity Long-Distance Depth Sensing](https://mlsanity.com/paper/2505.22025/) (2025)
- [ToFormer: Towards Large-scale Scenario Depth Completion for Lightweight ToF Camera](https://mlsanity.com/paper/2603.20669/) (2026)

## Abstract (arXiv)

Indirect Time-of-Flight (I-ToF) imaging is a widespread way of depth estimation for mobile devices due to its small size and affordable price. Previous works have mainly focused on quality improvement for I-ToF imaging especially curing the effect of Multi Path Interference (MPI). These investigations are typically done in specifically constrained scenarios at close distance, indoors and under little ambient light. Surprisingly little work has investigated I-ToF quality improvement in real-life scenarios where strong ambient light and far distances pose difficulties due to an extreme amount of induced shot noise and signal sparsity, caused by the attenuation with limited sensor power and light scattering. In this work, we propose a new learning based end-to-end depth prediction network which takes noisy raw I-ToF signals as well as an RGB image and fuses their latent representation based on a multi step approach involving both implicit and explicit alignment to predict a high quality long range depth map aligned to the RGB viewpoint. We test our approach on challenging real-world scenes and show more than 40% RMSE improvement on the final depth map compared to the baseline approach.