# RADU: Ray-Aligned Depth Update Convolutions for ToF Data Denoising

arXiv: https://arxiv.org/abs/2111.15513 · HTML: https://mlsanity.com/paper/2111.15513/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 44 references that are in the map.
- First posted on arXiv 30 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Thingi10K: A Dataset of 10,000 3D-Printing Models](https://mlsanity.com/paper/1605.04797/) (2016)
- [Understanding Self-Training for Gradual Domain Adaptation](https://mlsanity.com/paper/2002.11361/) (2020)
- [Cycle Self-Training for Domain Adaptation](https://mlsanity.com/paper/2103.03571/) (2021)
- [Patch-based Progressive 3D Point Set Upsampling](https://mlsanity.com/paper/1811.11286/) (2018)
- [PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds](https://mlsanity.com/paper/1901.01060/) (2019)
- [EC-Net: an Edge-aware Point set Consolidation Network](https://mlsanity.com/paper/1807.06010/) (2018)
- [SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2012.11460/) (2020)
- [Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds](https://mlsanity.com/paper/1806.01759/) (2018)
- [Total Denoising: Unsupervised Learning of 3D Point Cloud Cleaning](https://mlsanity.com/paper/1904.07615/) (2019)
- [In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness](https://mlsanity.com/paper/2012.04550/) (2020)
- [Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset](https://mlsanity.com/paper/1807.10376/) (2018)
- [Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module](https://mlsanity.com/paper/1909.07623/) (2019)
- [DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging](https://mlsanity.com/paper/1805.09305/) (2018)
- [Learning to Remove Multipath Distortions in Time-of-Flight Range Images for a Robotic Arm Setup](https://mlsanity.com/paper/1601.01750/) (2016)
- [iToF2dToF: A Robust and Flexible Representation for Data-Driven Time-of-Flight Imaging](https://mlsanity.com/paper/2103.07087/) (2021)
- [SRA: Fast Removal of General Multipath for ToF Sensors](https://mlsanity.com/paper/1403.5919/) (2014)
- [Very Power Efficient Neural Time-of-Flight](https://mlsanity.com/paper/1812.08125/) (2018)
- [Depth Reconstruction of Translucent Objects from a Single Time-of-Flight Camera using Deep Residual Networks](https://mlsanity.com/paper/1809.10917/) (2018)

## Built on by (most cited first, 5 of 5)

- [Self-Annotated 3D Geometric Learning for Smeared Points Removal](https://mlsanity.com/paper/2311.09029/) (2023)
- [Weakly-Supervised Optical Flow Estimation for Time-of-Flight](https://mlsanity.com/paper/2210.05298/) (2022)
- [Consistent Time-of-Flight Depth Denoising via Graph-Informed Geometric Attention](https://mlsanity.com/paper/2506.23542/) (2025)
- [Towards Robust Time-of-Flight Depth Denoising with Confidence-Aware Diffusion Model](https://mlsanity.com/paper/2503.19448/) (2025)
- [Reliable iToF Depth Sensing via Sensor-Intrinsic Uncertainty Modeling and State-Space Restoration](https://mlsanity.com/paper/2609.05507/) (2026)

## Abstract (arXiv)

Time-of-Flight (ToF) cameras are subject to high levels of noise and distortions due to Multi-Path-Interference (MPI). While recent research showed that 2D neural networks are able to outperform previous traditional State-of-the-Art (SOTA) methods on denoising ToF-Data, little research on learning-based approaches has been done to make direct use of the 3D information present in depth images. In this paper, we propose an iterative denoising approach operating in 3D space, that is designed to learn on 2.5D data by enabling 3D point convolutions to correct the points' positions along the view direction. As labeled real world data is scarce for this task, we further train our network with a self-training approach on unlabeled real world data to account for real world statistics. We demonstrate that our method is able to outperform SOTA methods on several datasets, including two real world datasets and a new large-scale synthetic data set introduced in this paper.