# Centimeter-Wave Free-Space Time-of-Flight Imaging

arXiv: https://arxiv.org/abs/2105.11606 · HTML: https://mlsanity.com/paper/2105.11606/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 103 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Fast-SCNN: Fast Semantic Segmentation Network](https://mlsanity.com/paper/1902.04502/) (2019)
- [ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans](https://mlsanity.com/paper/1712.10215/) (2017)
- [Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Prediction](https://mlsanity.com/paper/1801.03910/) (2018)
- [Learning Single Camera Depth Estimation using Dual-Pixels](https://mlsanity.com/paper/1904.05822/) (2019)
- [Asynchronous Single-Photon 3D Imaging](https://mlsanity.com/paper/1908.06372/) (2019)
- [Photon-Flooded Single-Photon 3D Cameras](https://mlsanity.com/paper/1903.08347/) (2019)
- [Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset](https://mlsanity.com/paper/1807.10376/) (2018)
- [On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach](https://mlsanity.com/paper/1803.09719/) (2018)
- [Self-Supervised Deep Depth Denoising](https://mlsanity.com/paper/1909.01193/) (2019)
- [Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module](https://mlsanity.com/paper/1909.07623/) (2019)
- [SRA: Fast Removal of General Multipath for ToF Sensors](https://mlsanity.com/paper/1403.5919/) (2014)
- [Resolving Multi-path Interference in Time-of-Flight Imaging via Modulation Frequency Diversity and Sparse Regularization](https://mlsanity.com/paper/1404.1116/) (2014)
- [Efficient Hierarchical Graph-Based Segmentation of RGBD Videos](https://mlsanity.com/paper/1801.08981/) (2018)
- [Efficient Multi-Frequency Phase Unwrapping using Kernel Density Estimation](https://mlsanity.com/paper/1608.05209/) (2016)
- [Snapshot Difference Imaging using Time-of-Flight Sensors](https://mlsanity.com/paper/1705.07108/) (2017)

## Abstract (arXiv)

Depth cameras are emerging as a cornerstone modality with diverse applications that directly or indirectly rely on measured depth, including personal devices, robotics, and self-driving vehicles. Although time-of-flight (ToF) methods have fueled these applications, the precision and robustness of ToF methods is limited by relying on photon time-tagging or modulation after photo-conversion. Successful optical modulation approaches have been restricted fiber-coupled modulation with large coupling losses or interferometric modulation with sub-cm range, and the precision gap between interferometric methods and ToF methods is more than three orders of magnitudes. In this work, we close this gap and propose a computational imaging method for all-optical free-space correlation before photo-conversion that achieves micron-scale depth resolution with robustness to surface reflectance and ambient light with conventional silicon intensity sensors. To this end, we solve two technical challenges: modulating at GHz rates and computational phase unwrapping. We propose an imaging approach with resonant polarization modulators and devise a novel optical dual-pass frequency-doubling which achieves high modulation contrast at more than 10GHz. At the same time, centimeter-wave modulation together with a small modulation bandwidth render existing phase unwrapping methods ineffective. We tackle this problem with a neural phase unwrapping method that exploits that adjacent wraps are often highly correlated. We validate the proposed method in simulation and experimentally, where it achieves micron-scale depth precision. We demonstrate precise depth sensing independently of surface texture and ambient light and compare against existing analog demodulation methods, which we outperform across all tested scenarios.