mlsanityData to 7 Oct 2026

Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion

arXiv 2103.16690 ↗First posted 30 Mar 2021cs.CV cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

Estimating scene geometry from data obtained with cost-effective sensors is key for robots and self-driving cars. In this paper, we study the problem of predicting dense depth from a single RGB image (monodepth) with optional sparse measurements from low-cost active depth sensors. We introduce Sparse Auxiliary Networks (SANs), a new module enabling monodepth networks to perform both the tasks of depth prediction and completion, depending on whether only RGB images or also sparse point clouds are available at inference time. First, we decouple the image and depth map encoding stages using sparse convolutions to process only the valid depth map pixels. Second, we inject this information, when available, into the skip connections of the depth prediction network, augmenting its features. Through extensive experimental analysis on one indoor (NYUv2) and two outdoor (KITTI and DDAD) benchmarks, we demonstrate that our proposed SAN architecture is able to simultaneously learn both tasks, while achieving a new state of the art in depth prediction by a significant margin.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 30 Mar 2021.

Builds on

43 of 57 references.

References · 43

  1. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  4. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  5. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  6. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  7. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  8. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  9. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  10. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  11. 2017Sparsity Invariant CNNscited by 325
  12. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
31 more
  1. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  2. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  3. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  4. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  5. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  6. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  7. 2018Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Cameracited by 140
  8. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  9. 2018High Quality Monocular Depth Estimation via Transfer Learningcited by 133
  10. 2018DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Imagecited by 129
  11. 2019Learning Guided Convolutional Network for Depth Completioncited by 111
  12. 2017Semi-Supervised Deep Learning for Monocular Depth Map Predictioncited by 109
  13. 2018Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Networkcited by 109
  14. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  15. 2016Learning Dense Correspondence via 3D-guided Cycle Consistencycited by 91
  16. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  17. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  18. 2019Sparse and noisy LiDAR completion with RGB guidance and uncertaintycited by 79
  19. 2020Learning multiview 3D point cloud registrationcited by 78
  20. 2018Learning Monocular Depth by Distilling Cross-domain Stereo Networkscited by 64
  21. 2018Confidence Propagation through CNNs for Guided Sparse Depth Regressioncited by 52
  22. 2020Probabilistic Future Prediction for Video Scene Understandingcited by 45
  23. 2019Depth Coefficients for Depth Completioncited by 41
  24. 2020Distilled Semantics for Comprehensive Scene Understanding from Videoscited by 39
  25. 2019Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distancescited by 27
  26. 2019Unsupervised High-Resolution Depth Learning From Videos With Dual Networkscited by 26
  27. 2019RefinedMPL: Refined Monocular PseudoLiDAR for 3D Object Detection in Autonomous Drivingcited by 23
  28. 2019Self-Supervised 3D Keypoint Learning for Ego-motion Estimationcited by 12
  29. 2019SDNet: Semantically Guided Depth Estimation Networkcited by 11
  30. 2020Footprints and Free Space from a Single Color Imagecited by 6
  31. 2019DFineNet: Ego-Motion Estimation and Depth Refinement from Sparse, Noisy Depth Input with RGB Guidancecited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 16; 2024: 8; 2025: 5; 2026: 1.

39 papers.

Sort:

By area:

  1. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151
  2. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  3. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 92
  4. 2023CompletionFormer: Depth Completion with Convolutions and Vision Transformerscited by 92
  5. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  6. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 81
  7. 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimationcited by 45
  8. 2022Sparsity Agnostic Depth Completioncited by 27
  9. 2023GEDepth: Ground Embedding for Monocular Depth Estimationcited by 26
  10. 2024Depth Prompting for Sensor-Agnostic Depth Estimationcited by 26
  11. 2023URCDC-Depth: Uncertainty Rectified Cross-Distillation with CutFlip for Monocular Depth Estimationcited by 16
  12. 2025Vision-Language Embodiment for Monocular Depth Estimationcited by 16
27 more
  1. 2022The Monocular Depth Estimation Challengecited by 13
  2. 2023G2-MonoDepth: A General Framework of Generalized Depth Inference from Monocular RGB+X Datacited by 12
  3. 2024DoubleTake: Geometry Guided Depth Estimationcited by 9
  4. 2022Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimationcited by 8
  5. 2023Diffusion-Augmented Depth Prediction with Sparse Annotationscited by 8
  6. 2023NDDepth: Normal-Distance Assisted Monocular Depth Estimation and Completioncited by 8
  7. 2024RGB Guided ToF Imaging System: A Survey of Deep Learning-based Methodscited by 8
  8. 2021LiDARTouch: Monocular metric depth estimation with a few-beam LiDARcited by 7
  9. 2022A Low Memory Footprint Quantized Neural Network for Depth Completion of Very Sparse Time-of-Flight Depth Mapscited by 6
  10. 2024GroCo: Ground Constraint for Metric Self-Supervised Monocular Depthcited by 6
  11. 2023Continual Learning of Unsupervised Monocular Depth from Videoscited by 5
  12. 2024Masked Spatial Propagation Network for Sparsity-Adaptive Depth Refinementcited by 5
  13. 2023Predictive World Models from Real-World Partial Observationscited by 4
  14. 2025Deep Depth Estimation from Thermal Image: Dataset, Benchmark, and Challengescited by 4
  15. 2025PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistencycited by 4
  16. 2023Meta-Optimization for Higher Model Generalizability in Single-Image Depth Predictioncited by 3
  17. 2025Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learningcited by 3
  18. 2022Positional Information is All You Need: A Novel Pipeline for Self-Supervised SVDE from Videoscited by 2
  19. 2023ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimationcited by 2
  20. 2024RoofDiffusion: Constructing Roofs from Severely Corrupted Point Data via Diffusioncited by 2
  21. 2023Do More With What You Have: Transferring Depth-Scale from Labeled to Unlabeled Domainscited by 1
  22. 2023LiDAR Meta Depth Completioncited by 1
  23. 2024Boosting Generalizability towards Zero-Shot Cross-Dataset Single-Image Indoor Depth by Meta-Initializationcited by 1
  24. 2025Dream-to-Recon: Monocular 3D Reconstruction with Diffusion-Depth Distillation from Single Imagescited by 1
  25. 2023Learning Monocular Depth in Dynamic Environment via Context-aware Temporal Attentioncited by 0
  26. 2024KineDepth: Utilizing Robot Kinematics for Online Metric Depth Estimationcited by 0
  27. 2026Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Predictioncited by 0