# Towards Robust Monocular Depth Estimation in Non-Lambertian Surfaces

arXiv: https://arxiv.org/abs/2408.06083 · HTML: https://mlsanity.com/paper/2408.06083/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 12 Aug 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization](https://mlsanity.com/paper/2403.06912/) (2024)
- [Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images](https://mlsanity.com/paper/2311.13398/) (2023)
- [Towards Zero-Shot Scale-Aware Monocular Depth Estimation](https://mlsanity.com/paper/2306.17253/) (2023)
- [DepthFM: Fast Monocular Depth Estimation with Flow Matching](https://mlsanity.com/paper/2403.13788/) (2024)
- [Open Challenges in Deep Stereo: the Booster Dataset](https://mlsanity.com/paper/2206.04671/) (2022)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces](https://mlsanity.com/paper/2301.08245/) (2023)
- [Dense 3D Object Reconstruction from a Single Depth View](https://mlsanity.com/paper/1802.00411/) (2018)
- [3D Cinemagraphy from a Single Image](https://mlsanity.com/paper/2303.05724/) (2023)
- [Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Image](https://mlsanity.com/paper/2308.10257/) (2023)
- [Mirror3D: Depth Refinement for Mirror Surfaces](https://mlsanity.com/paper/2106.06629/) (2021)
- [Symmetry-Aware Transformer-based Mirror Detection](https://mlsanity.com/paper/2207.06332/) (2022)
- [Diffusion-Augmented Depth Prediction with Sparse Annotations](https://mlsanity.com/paper/2308.02283/) (2023)
- [Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report](https://mlsanity.com/paper/2211.04470/) (2022)

## Built on by (most cited first, 9 of 9)

- [TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion](https://mlsanity.com/paper/2504.11773/) (2025)
- [Self-Distilled Depth Refinement with Noisy Poisson Fusion](https://mlsanity.com/paper/2409.17880/) (2024)
- [BokehFlow: Depth-Free Controllable Bokeh Rendering via Flow Matching](https://mlsanity.com/paper/2511.15066/) (2025)
- [R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video](https://mlsanity.com/paper/2608.11017/) (2026)
- [SinkSAM-Net: Knowledge-Driven Self-Supervised Sinkhole Segmentation Using Topographic Priors and Segment Anything Model](https://mlsanity.com/paper/2410.01473/) (2024)
- [Seamless High-Resolution Terrain Reconstruction: A Prior-Based Vision Transformer Approach](https://mlsanity.com/paper/2507.09681/) (2025)
- [SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping](https://mlsanity.com/paper/2605.28735/) (2026)
- [Beyond Visual Ambiguity: Guiding Robust Monocular Depth Estimation in Challenging Scenarios via Detailed Long Captions](https://mlsanity.com/paper/2607.28285/) (2026)
- [GIFT: Geometry-Invariant Fine-Tuning for Non-Lambertian Monocular Depth Estimation](https://mlsanity.com/paper/2608.02068/) (2026)

## Abstract (arXiv)

In the field of monocular depth estimation (MDE), many models with excellent zero-shot performance in general scenes emerge recently. However, these methods often fail in predicting non-Lambertian surfaces, such as transparent or mirror (ToM) surfaces, due to the unique reflective properties of these regions. Previous methods utilize externally provided ToM masks and aim to obtain correct depth maps through direct in-painting of RGB images. These methods highly depend on the accuracy of additional input masks, and the use of random colors during in-painting makes them insufficiently robust. We are committed to incrementally enabling the baseline model to directly learn the uniqueness of non-Lambertian surface regions for depth estimation through a well-designed training framework. Therefore, we propose non-Lambertian surface regional guidance, which constrains the predictions of MDE model from the gradient domain to enhance its robustness. Noting the significant impact of lighting on this task, we employ the random tone-mapping augmentation during training to ensure the network can predict correct results for varying lighting inputs. Additionally, we propose an optional novel lighting fusion module, which uses Variational Autoencoders to fuse multiple images and obtain the most advantageous input RGB image for depth estimation when multi-exposure images are available. Our method achieves accuracy improvements of 33.39% and 5.21% in zero-shot testing on the Booster and Mirror3D dataset for non-Lambertian surfaces, respectively, compared to the Depth Anything V2. The state-of-the-art performance of 90.75 in delta1.05 within the ToM regions on the TRICKY2024 competition test set demonstrates the effectiveness of our approach.