# BP-MVSNet: Belief-Propagation-Layers for Multi-View-Stereo

arXiv: https://arxiv.org/abs/2010.12436 · HTML: https://mlsanity.com/paper/2010.12436/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth](https://mlsanity.com/paper/2208.02541/) (2022)
- [Generalized Binary Search Network for Highly-Efficient Multi-View Stereo](https://mlsanity.com/paper/2112.02338/) (2021)
- [PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility](https://mlsanity.com/paper/2108.08943/) (2021)
- [Curvature-guided dynamic scale networks for Multi-view Stereo](https://mlsanity.com/paper/2112.05999/) (2021)
- [Non-parametric Depth Distribution Modelling based Depth Inference for Multi-view Stereo](https://mlsanity.com/paper/2205.03783/) (2022)
- [IB-MVS: An Iterative Algorithm for Deep Multi-View Stereo based on Binary Decisions](https://mlsanity.com/paper/2111.14420/) (2021)
- [PatchMVSNet: Patch-wise Unsupervised Multi-View Stereo for Weakly-Textured Surface Reconstruction](https://mlsanity.com/paper/2203.02156/) (2022)
- [DELS-MVS: Deep Epipolar Line Search for Multi-View Stereo](https://mlsanity.com/paper/2212.06626/) (2022)
- [Multi-View Stereo Representation Revisit: Region-Aware MVSNet](https://mlsanity.com/paper/2304.13614/) (2023)
- [Learning Deformable Hypothesis Sampling for Accurate PatchMatch Multi-View Stereo](https://mlsanity.com/paper/2312.15970/) (2023)

## Abstract (arXiv)

In this work, we propose BP-MVSNet, a convolutional neural network (CNN)-based Multi-View-Stereo (MVS) method that uses a differentiable Conditional Random Field (CRF) layer for regularization. To this end, we propose to extend the BP layer and add what is necessary to successfully use it in the MVS setting. We therefore show how we can calculate a normalization based on the expected 3D error, which we can then use to normalize the label jumps in the CRF. This is required to make the BP layer invariant to different scales in the MVS setting. In order to also enable fractional label jumps, we propose a differentiable interpolation step, which we embed into the computation of the pairwise term. These extensions allow us to integrate the BP layer into a multi-scale MVS network, where we continuously improve a rough initial estimate until we get high quality depth maps as a result. We evaluate the proposed BP-MVSNet in an ablation study and conduct extensive experiments on the DTU, Tanks and Temples and ETH3D data sets. The experiments show that we can significantly outperform the baseline and achieve state-of-the-art results.