# FaSS-MVS -- Fast Multi-View Stereo with Surface-Aware Semi-Global Matching from UAV-borne Monocular Imagery

arXiv: https://arxiv.org/abs/2112.00821 · HTML: https://mlsanity.com/paper/2112.00821/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 70 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [Pyramid Multi-view Stereo Net with Self-adaptive View Aggregation](https://mlsanity.com/paper/1912.03001/) (2019)
- [Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency](https://mlsanity.com/paper/1905.02706/) (2019)
- [M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network](https://mlsanity.com/paper/2005.00363/) (2020)
- [Learned Multi-Patch Similarity](https://mlsanity.com/paper/1703.08836/) (2017)
- [Embedded real-time stereo estimation via Semi-Global Matching on the GPU](https://mlsanity.com/paper/1610.04121/) (2016)
- [FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications](https://mlsanity.com/paper/2006.03250/) (2020)
- [Efficient Surface-Aware Semi-Global Matching with Multi-View Plane-Sweep Sampling](https://mlsanity.com/paper/1909.09891/) (2019)
- [Real-time dense 3D Reconstruction from monocular video data captured by low-cost UAVs](https://mlsanity.com/paper/2104.10515/) (2021)
- [ReS2tAC -- UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices](https://mlsanity.com/paper/2106.07927/) (2021)
- [Semi-Global Stereo Matching with Surface Orientation Priors](https://mlsanity.com/paper/1712.00818/) (2017)

## Abstract (arXiv)

With FaSS-MVS, we present an approach for fast multi-view stereo with surface-aware Semi-Global Matching that allows for rapid depth and normal map estimation from monocular aerial video data captured by UAVs. The data estimated by FaSS-MVS, in turn, facilitates online 3D mapping, meaning that a 3D map of the scene is immediately and incrementally generated while the image data is acquired or being received. FaSS-MVS is comprised of a hierarchical processing scheme in which depth and normal data, as well as corresponding confidence scores, are estimated in a coarse-to-fine manner, allowing to efficiently process large scene depths which are inherent to oblique imagery captured by low-flying UAVs. The actual depth estimation employs a plane-sweep algorithm for dense multi-image matching to produce depth hypotheses from which the actual depth map is extracted by means of a surface-aware semi-global optimization, reducing the fronto-parallel bias of SGM. Given the estimated depth map, the pixel-wise surface normal information is then computed by reprojecting the depth map into a point cloud and calculating the normal vectors within a confined local neighborhood. In a thorough quantitative and ablative study we show that the accuracies of the 3D information calculated by FaSS-MVS is close to that of state-of-the-art approaches for offline multi-view stereo, with the error not even being one magnitude higher than that of COLMAP. At the same time, however, the average run-time of FaSS-MVS to estimate a single depth and normal map is less than 14 % of that of COLMAP, allowing to perform an online and incremental processing of Full-HD imagery at 1-2 Hz.