# Adaptive Illumination based Depth Sensing using Deep Superpixel and Soft Sampling Approximation

arXiv: https://arxiv.org/abs/2103.12297 · HTML: https://mlsanity.com/paper/2103.12297/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 74 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 30 of 31)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [High Quality Monocular Depth Estimation via Transfer Learning](https://mlsanity.com/paper/1812.11941/) (2018)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [FastDepth: Fast Monocular Depth Estimation on Embedded Systems](https://mlsanity.com/paper/1903.03273/) (2019)
- [Unsupervised Depth Completion from Visual Inertial Odometry](https://mlsanity.com/paper/1905.08616/) (2019)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Superpixel Sampling Networks](https://mlsanity.com/paper/1807.10174/) (2018)
- [Sparse and noisy LiDAR completion with RGB guidance and uncertainty](https://mlsanity.com/paper/1902.05356/) (2019)
- [Sparse and Dense Data with CNNs: Depth Completion and Semantic Segmentation](https://mlsanity.com/paper/1808.00769/) (2018)
- [Superpixel Segmentation with Fully Convolutional Networks](https://mlsanity.com/paper/2003.12929/) (2020)
- [DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling](https://mlsanity.com/paper/2001.00987/) (2019)
- [Dense Depth Posterior (DDP) from Single Image and Sparse Range](https://mlsanity.com/paper/1901.10034/) (2019)
- [DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion](https://mlsanity.com/paper/1902.00761/) (2019)
- [Estimating Depth from RGB and Sparse Sensing](https://mlsanity.com/paper/1804.02771/) (2018)
- [Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging](https://mlsanity.com/paper/1808.03571/) (2018)
- [Data-Driven Design for Fourier Ptychographic Microscopy](https://mlsanity.com/paper/1904.04175/) (2019)
- [Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling](https://mlsanity.com/paper/1806.10175/) (2018)
- [Adaptive Image Sampling using Deep Learning and its Application on X-Ray Fluorescence Image Reconstruction](https://mlsanity.com/paper/1812.10836/) (2018)
- [Learning Sampling and Model-Based Signal Recovery for Compressed Sensing MRI](https://mlsanity.com/paper/2004.10536/) (2020)
- [Sparse Depth Sensing for Resource-Constrained Robots](https://mlsanity.com/paper/1703.01398/) (2017)

## Built on by (most cited first, 2 of 2)

- [Progressive Depth Decoupling and Modulating for Flexible Depth Completion](https://mlsanity.com/paper/2405.09342/) (2024)
- [Computer Vision with a Superpixelation Camera](https://mlsanity.com/paper/2603.26900/) (2026)

## Abstract (arXiv)

Dense depth map capture is challenging in existing active sparse illumination based depth acquisition techniques, such as LiDAR. Various techniques have been proposed to estimate a dense depth map based on fusion of the sparse depth map measurement with the RGB image. Recent advances in hardware enable adaptive depth measurements resulting in further improvement of the dense depth map estimation. In this paper, we study the topic of estimating dense depth from depth sampling. The adaptive sparse depth sampling network is jointly trained with a fusion network of an RGB image and sparse depth, to generate optimal adaptive sampling masks. We show that such adaptive sampling masks can generalize well to many RGB and sparse depth fusion algorithms under a variety of sampling rates (as low as 0.0625%). The proposed adaptive sampling method is fully differentiable and flexible to be trained end-to-end with upstream perception algorithms.