# Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry

arXiv: https://arxiv.org/abs/2112.08177 · HTML: https://mlsanity.com/paper/2112.08177/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 69 references that are in the map.
- First posted on arXiv 15 Dec 2021.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [DeepV2D: Video to Depth with Differentiable Structure from Motion](https://mlsanity.com/paper/1812.04605/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)

## Built on by (most cited first, 30 of 46)

- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [DepthSplat: Connecting Gaussian Splatting and Depth](https://mlsanity.com/paper/2410.13862/) (2024)
- [Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection](https://mlsanity.com/paper/2210.02443/) (2022)
- [BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereo](https://mlsanity.com/paper/2209.10248/) (2022)
- [STS: Surround-view Temporal Stereo for Multi-view 3D Detection](https://mlsanity.com/paper/2208.10145/) (2022)
- [Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving](https://mlsanity.com/paper/2403.07535/) (2024)
- [Cross-View Completion Models are Zero-shot Correspondence Estimators](https://mlsanity.com/paper/2412.09072/) (2024)
- [D-IF: Uncertainty-aware Human Digitization via Implicit Distribution Field](https://mlsanity.com/paper/2308.08857/) (2023)
- [Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2304.08993/) (2023)
- [PanoGRF: Generalizable Spherical Radiance Fields for Wide-baseline Panoramas](https://mlsanity.com/paper/2306.01531/) (2023)
- [SPARC: Sparse Render-and-Compare for CAD model alignment in a single RGB image](https://mlsanity.com/paper/2210.01044/) (2022)
- [MonSter++: Unified Stereo Matching, Multi-view Stereo, and Real-time Stereo with Monodepth Priors](https://mlsanity.com/paper/2501.08643/) (2025)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [BridgeDepth: Bridging Monocular and Stereo Reasoning with Latent Alignment](https://mlsanity.com/paper/2508.04611/) (2025)
- [ImGeoNet: Image-induced Geometry-aware Voxel Representation for Multi-view 3D Object Detection](https://mlsanity.com/paper/2308.09098/) (2023)
- [Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning](https://mlsanity.com/paper/2208.09170/) (2022)
- [SfM-TTR: Using Structure from Motion for Test-Time Refinement of Single-View Depth Networks](https://mlsanity.com/paper/2211.13551/) (2022)
- [IMP: Iterative Matching and Pose Estimation with Adaptive Pooling](https://mlsanity.com/paper/2304.14837/) (2023)
- [M2Depth: Self-supervised Two-Frame Multi-camera Metric Depth Estimation](https://mlsanity.com/paper/2405.02004/) (2024)
- [Diving into the Fusion of Monocular Priors for Generalized Stereo Matching](https://mlsanity.com/paper/2505.14414/) (2025)
- [GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mapping](https://mlsanity.com/paper/2205.01656/) (2022)
- [Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy](https://mlsanity.com/paper/2311.18664/) (2023)
- [MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps](https://mlsanity.com/paper/2410.21566/) (2024)
- [OpenM3D: Open Vocabulary Multi-view Indoor 3D Object Detection without Human Annotations](https://mlsanity.com/paper/2508.20063/) (2025)
- [Predict to Detect: Prediction-guided 3D Object Detection using Sequential Images](https://mlsanity.com/paper/2306.08528/) (2023)
- [Learning Feature Matching via Matchable Keypoint-Assisted Graph Neural Network](https://mlsanity.com/paper/2307.01447/) (2023)
- [TempBEV: Improving Learned BEV Encoders with Combined Image and BEV Space Temporal Aggregation](https://mlsanity.com/paper/2404.11803/) (2024)
- [UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images](https://mlsanity.com/paper/2602.24290/) (2026)
- [BEVStereo++: Accurate Depth Estimation in Multi-view 3D Object Detection via Dynamic Temporal Stereo](https://mlsanity.com/paper/2304.04185/) (2023)
- [CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning](https://mlsanity.com/paper/2503.08219/) (2025)

## Abstract (arXiv)

Multi-view depth estimation methods typically require the computation of a multi-view cost-volume, which leads to huge memory consumption and slow inference. Furthermore, multi-view matching can fail for texture-less surfaces, reflective surfaces and moving objects. For such failure modes, single-view depth estimation methods are often more reliable. To this end, we propose MaGNet, a novel framework for fusing single-view depth probability with multi-view geometry, to improve the accuracy, robustness and efficiency of multi-view depth estimation. For each frame, MaGNet estimates a single-view depth probability distribution, parameterized as a pixel-wise Gaussian. The distribution estimated for the reference frame is then used to sample per-pixel depth candidates. Such probabilistic sampling enables the network to achieve higher accuracy while evaluating fewer depth candidates. We also propose depth consistency weighting for the multi-view matching score, to ensure that the multi-view depth is consistent with the single-view predictions. The proposed method achieves state-of-the-art performance on ScanNet, 7-Scenes and KITTI. Qualitative evaluation demonstrates that our method is more robust against challenging artifacts such as texture-less/reflective surfaces and moving objects. Our code and model weights are available at https://github.com/baegwangbin/MaGNet.