# Revisiting PatchMatch Multi-View Stereo for Urban 3D Reconstruction

arXiv: https://arxiv.org/abs/2207.08439 · HTML: https://mlsanity.com/paper/2207.08439/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 37 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Multi-Scale Geometric Consistency Guided Multi-View Stereo](https://mlsanity.com/paper/1904.08103/) (2019)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [Planar Prior Assisted PatchMatch Multi-View Stereo](https://mlsanity.com/paper/1912.11744/) (2019)
- [LEGO: Learning Edge with Geometry all at Once by Watching Videos](https://mlsanity.com/paper/1803.05648/) (2018)
- [TAPA-MVS: Textureless-Aware PAtchMatch Multi-View Stereo](https://mlsanity.com/paper/1903.10929/) (2019)
- [Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances](https://mlsanity.com/paper/1910.01765/) (2019)
- [DeepC-MVS: Deep Confidence Prediction for Multi-View Stereo Reconstruction](https://mlsanity.com/paper/1912.00439/) (2019)
- [Boosting Monocular Depth Estimation with Lightweight 3D Point Fusion](https://mlsanity.com/paper/2012.10296/) (2020)
- [Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR](https://mlsanity.com/paper/2109.09628/) (2021)
- [Joint Graph-based Depth Refinement and Normal Estimation](https://mlsanity.com/paper/1912.01306/) (2019)
- [Self-supervised Learning for Single View Depth and Surface Normal Estimation](https://mlsanity.com/paper/1903.00112/) (2019)
- [Multi-View Stereo with Asymmetric Checkerboard Propagation and Multi-Hypothesis Joint View Selection](https://mlsanity.com/paper/1805.07920/) (2018)

## Built on by (most cited first, 3 of 3)

- [Learning Neural Radiance Fields from Multi-View Geometry](https://mlsanity.com/paper/2210.13041/) (2022)
- [Informative Rays Selection for Few-Shot Neural Radiance Fields](https://mlsanity.com/paper/2312.17561/) (2023)
- [City-Mesh3R: Simulation-Ready City-Scale 3D Mesh Reconstruction from Multi-View Images](https://mlsanity.com/paper/2605.30310/) (2026)

## Abstract (arXiv)

In this paper, a complete pipeline for image-based 3D reconstruction of urban scenarios is proposed, based on PatchMatch Multi-View Stereo (MVS). Input images are firstly fed into an off-the-shelf visual SLAM system to extract camera poses and sparse keypoints, which are used to initialize PatchMatch optimization. Then, pixelwise depths and normals are iteratively computed in a multi-scale framework with a novel depth-normal consistency loss term and a global refinement algorithm to balance the inherently local nature of PatchMatch. Finally, a large-scale point cloud is generated by back-projecting multi-view consistent estimates in 3D. The proposed approach is carefully evaluated against both classical MVS algorithms and monocular depth networks on the KITTI dataset, showing state of the art performances.