# Automatic calibration of time of flight based non-line-of-sight reconstruction

arXiv: https://arxiv.org/abs/2105.10603 · HTML: https://mlsanity.com/paper/2105.10603/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 42 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Virtual Wave Optics for Non-Line-of-Sight Imaging](https://mlsanity.com/paper/1810.07535/) (2018)
- [Learning Sensor Multiplexing Design through Back-propagation](https://mlsanity.com/paper/1605.07078/) (2016)
- [Recent Advances in Imaging Around Corners](https://mlsanity.com/paper/1910.05613/) (2019)
- [Flash Photography for Data-Driven Hidden Scene Recovery](https://mlsanity.com/paper/1810.11710/) (2018)
- [Efficient illumination angle self-calibration in Fourier ptychography](https://mlsanity.com/paper/1804.03299/) (2018)
- [Towards an Intelligent Microscope: adaptively learned illumination for optimal sample classification](https://mlsanity.com/paper/1910.10209/) (2019)

## Abstract (arXiv)

Time of flight based Non-line-of-sight (NLOS) imaging approaches require precise calibration of illumination and detector positions on the visible scene to produce reasonable results. If this calibration error is sufficiently high, reconstruction can fail entirely without any indication to the user. In this work, we highlight the necessity of building autocalibration into NLOS reconstruction in order to handle mis-calibration. We propose a forward model of NLOS measurements that is differentiable with respect to both, the hidden scene albedo, and virtual illumination and detector positions. With only a mean squared error loss and no regularization, our model enables joint reconstruction and recovery of calibration parameters by minimizing the measurement residual using gradient descent. We demonstrate our method is able to produce robust reconstructions using simulated and real data where the calibration error applied causes other state of the art algorithms to fail.