# Occlusion-Aware Depth Estimation with Adaptive Normal Constraints

arXiv: https://arxiv.org/abs/2004.00845 · HTML: https://mlsanity.com/paper/2004.00845/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 30 of 41)

- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2206.05737/) (2022)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [Neural 3D Scene Reconstruction with the Manhattan-world Assumption](https://mlsanity.com/paper/2205.02836/) (2022)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies](https://mlsanity.com/paper/2211.14173/) (2022)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [Self-supervised Monocular Depth Estimation for All Day Images using Domain Separation](https://mlsanity.com/paper/2108.07628/) (2021)
- [Adaptive Surface Normal Constraint for Depth Estimation](https://mlsanity.com/paper/2103.15483/) (2021)
- [MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions](https://mlsanity.com/paper/2104.13325/) (2021)
- [VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion](https://mlsanity.com/paper/2112.00236/) (2021)
- [Instance-wise Occlusion and Depth Orders in Natural Scenes](https://mlsanity.com/paper/2111.14562/) (2021)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving](https://mlsanity.com/paper/2311.16945/) (2023)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [Depth Field Networks for Generalizable Multi-view Scene Representation](https://mlsanity.com/paper/2207.14287/) (2022)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [Learnable Differencing Center for Nighttime Depth Perception](https://mlsanity.com/paper/2306.14538/) (2023)
- [Adaptive Surface Normal Constraint for Geometric Estimation from Monocular Images](https://mlsanity.com/paper/2402.05869/) (2024)
- [A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo](https://mlsanity.com/paper/2201.07609/) (2022)
- [Facial Depth and Normal Estimation using Single Dual-Pixel Camera](https://mlsanity.com/paper/2111.12928/) (2021)
- [PlaneRecNet: Multi-Task Learning with Cross-Task Consistency for Piece-Wise Plane Detection and Reconstruction from a Single RGB Image](https://mlsanity.com/paper/2110.11219/) (2021)
- [MonoSelfRecon: Purely Self-Supervised Explicit Generalizable 3D Reconstruction of Indoor Scenes from Monocular RGB Views](https://mlsanity.com/paper/2404.06753/) (2024)
- [Virtual Occlusions Through Implicit Depth](https://mlsanity.com/paper/2305.07014/) (2023)

## Abstract (arXiv)

We present a new learning-based method for multi-frame depth estimation from a color video, which is a fundamental problem in scene understanding, robot navigation or handheld 3D reconstruction. While recent learning-based methods estimate depth at high accuracy, 3D point clouds exported from their depth maps often fail to preserve important geometric feature (e.g., corners, edges, planes) of man-made scenes. Widely-used pixel-wise depth errors do not specifically penalize inconsistency on these features. These inaccuracies are particularly severe when subsequent depth reconstructions are accumulated in an attempt to scan a full environment with man-made objects with this kind of features. Our depth estimation algorithm therefore introduces a Combined Normal Map (CNM) constraint, which is designed to better preserve high-curvature features and global planar regions. In order to further improve the depth estimation accuracy, we introduce a new occlusion-aware strategy that aggregates initial depth predictions from multiple adjacent views into one final depth map and one occlusion probability map for the current reference view. Our method outperforms the state-of-the-art in terms of depth estimation accuracy, and preserves essential geometric features of man-made indoor scenes much better than other algorithms.