# S3: Learnable Sparse Signal Superdensity for Guided Depth Estimation

arXiv: https://arxiv.org/abs/2103.02396 · HTML: https://mlsanity.com/paper/2103.02396/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving](https://mlsanity.com/paper/1906.06310/) (2019)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [Learning Depth with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1810.02695/) (2018)
- [DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image](https://mlsanity.com/paper/1812.00488/) (2018)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [SEEDS: Superpixels Extracted via Energy-Driven Sampling](https://mlsanity.com/paper/1309.3848/) (2013)
- [Learning Joint 2D-3D Representations for Depth Completion](https://mlsanity.com/paper/2012.12402/) (2020)
- [A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection](https://mlsanity.com/paper/2005.07431/) (2020)
- [Distant Vehicle Detection Using Radar and Vision](https://mlsanity.com/paper/1901.10951/) (2019)
- [DFuseNet: Deep Fusion of RGB and Sparse Depth Information for Image Guided Dense Depth Completion](https://mlsanity.com/paper/1902.00761/) (2019)
- [Noise-Aware Unsupervised Deep Lidar-Stereo Fusion](https://mlsanity.com/paper/1904.03868/) (2019)
- [Indoor Depth Completion with Boundary Consistency and Self-Attention](https://mlsanity.com/paper/1908.08344/) (2019)
- [Guided Stereo Matching](https://mlsanity.com/paper/1905.10107/) (2019)
- [Deep RGB-D Canonical Correlation Analysis For Sparse Depth Completion](https://mlsanity.com/paper/1906.08967/) (2019)
- [3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization](https://mlsanity.com/paper/1904.02917/) (2019)
- [Real Time Dense Depth Estimation by Fusing Stereo with Sparse Depth Measurements](https://mlsanity.com/paper/1809.07677/) (2018)
- [300 GHz Radar Object Recognition based on Deep Neural Networks and Transfer Learning](https://mlsanity.com/paper/1912.03157/) (2019)

## Built on by (most cited first, 10 of 10)

- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization](https://mlsanity.com/paper/2312.09254/) (2023)
- [Active Stereo Without Pattern Projector](https://mlsanity.com/paper/2309.12315/) (2023)
- [LiDAR-Event Stereo Fusion with Hallucinations](https://mlsanity.com/paper/2408.04633/) (2024)
- [SST: Real-time End-to-end Monocular 3D Reconstruction via Sparse Spatial-Temporal Guidance](https://mlsanity.com/paper/2212.06524/) (2022)
- [Stereo-LiDAR Depth Estimation with Deformable Propagation and Learned Disparity-Depth Conversion](https://mlsanity.com/paper/2404.07545/) (2024)
- [Expansion of Visual Hints for Improved Generalization in Stereo Matching](https://mlsanity.com/paper/2211.00392/) (2022)
- [Depth-Relative Self Attention for Monocular Depth Estimation](https://mlsanity.com/paper/2304.12849/) (2023)
- [Active Stereo in the Wild through Virtual Pattern Projection](https://mlsanity.com/paper/2406.04345/) (2024)
- [Preserving Guidance in Cost-Volume Retrieval under Extreme LiDAR Sparsity in Iterative Stereo](https://mlsanity.com/paper/2507.19738/) (2025)

## Abstract (arXiv)

Dense depth estimation plays a key role in multiple applications such as robotics, 3D reconstruction, and augmented reality. While sparse signal, e.g., LiDAR and Radar, has been leveraged as guidance for enhancing dense depth estimation, the improvement is limited due to its low density and imbalanced distribution. To maximize the utility from the sparse source, we propose S3 technique, which expands the depth value from sparse cues while estimating the confidence of expanded region. The proposed S3 can be applied to various guided depth estimation approaches and trained end-to-end at different stages, including input, cost volume and output. Extensive experiments demonstrate the effectiveness, robustness, and flexibility of the S3 technique on LiDAR and Radar signal.