# Non-parametric Depth Distribution Modelling based Depth Inference for Multi-view Stereo

arXiv: https://arxiv.org/abs/2205.03783 · HTML: https://mlsanity.com/paper/2205.03783/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 8 May 2022.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution](https://mlsanity.com/paper/2007.16100/) (2020)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [Visibility-aware Multi-view Stereo Network](https://mlsanity.com/paper/2008.07928/) (2020)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network](https://mlsanity.com/paper/2108.03824/) (2021)
- [PVSNet: Pixelwise Visibility-Aware Multi-View Stereo Network](https://mlsanity.com/paper/2007.07714/) (2020)
- [Self-supervised Learning of Depth Inference for Multi-view Stereo](https://mlsanity.com/paper/2104.02972/) (2021)
- [PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility](https://mlsanity.com/paper/2108.08943/) (2021)
- [BP-MVSNet: Belief-Propagation-Layers for Multi-View-Stereo](https://mlsanity.com/paper/2010.12436/) (2020)

## Built on by (most cited first, 15 of 15)

- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching](https://mlsanity.com/paper/2306.15612/) (2023)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [Improving Transformer-based Image Matching by Cascaded Capturing Spatially Informative Keypoints](https://mlsanity.com/paper/2303.02885/) (2023)
- [Stereo Risk: A Continuous Modeling Approach to Stereo Matching](https://mlsanity.com/paper/2407.03152/) (2024)
- [SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo](https://mlsanity.com/paper/2306.08648/) (2023)
- [CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning](https://mlsanity.com/paper/2503.08219/) (2025)
- [DELS-MVS: Deep Epipolar Line Search for Multi-View Stereo](https://mlsanity.com/paper/2212.06626/) (2022)
- [EfficientDreamer: High-Fidelity and Robust 3D Creation via Orthogonal-view Diffusion Prior](https://mlsanity.com/paper/2308.13223/) (2023)
- [ARAI-MVSNet: A multi-view stereo depth estimation network with adaptive depth range and depth interval](https://mlsanity.com/paper/2308.09022/) (2023)
- [ICG-MVSNet: Learning Intra-view and Cross-view Relationships for Guidance in Multi-View Stereo](https://mlsanity.com/paper/2503.21525/) (2025)
- [Radiant Triangle Soup with Soft Connectivity Forces for 3D Reconstruction and Novel View Synthesis](https://mlsanity.com/paper/2505.23642/) (2025)
- [V-FUSE: Volumetric Depth Map Fusion with Long-Range Constraints](https://mlsanity.com/paper/2308.08715/) (2023)
- [VAD-GS: Visibility-Aware Densification for 3D Gaussian Splatting in Dynamic Urban Scenes](https://mlsanity.com/paper/2510.09364/) (2025)
- [Learning Stable Canonical Worlds for Novel View Synthesis and Beyond](https://mlsanity.com/paper/2606.23027/) (2026)

## Abstract (arXiv)

Recent cost volume pyramid based deep neural networks have unlocked the potential of efficiently leveraging high-resolution images for depth inference from multi-view stereo. In general, those approaches assume that the depth of each pixel follows a unimodal distribution. Boundary pixels usually follow a multi-modal distribution as they represent different depths; Therefore, the assumption results in an erroneous depth prediction at the coarser level of the cost volume pyramid and can not be corrected in the refinement levels leading to wrong depth predictions. In contrast, we propose constructing the cost volume by non-parametric depth distribution modeling to handle pixels with unimodal and multi-modal distributions. Our approach outputs multiple depth hypotheses at the coarser level to avoid errors in the early stage. As we perform local search around these multiple hypotheses in subsequent levels, our approach does not maintain the rigid depth spatial ordering and, therefore, we introduce a sparse cost aggregation network to derive information within each volume. We evaluate our approach extensively on two benchmark datasets: DTU and Tanks & Temples. Our experimental results show that our model outperforms existing methods by a large margin and achieves superior performance on boundary regions. Code is available at https://github.com/NVlabs/NP-CVP-MVSNet