# NeuralMVS: Bridging Multi-View Stereo and Novel View Synthesis

arXiv: https://arxiv.org/abs/2108.03880 · HTML: https://mlsanity.com/paper/2108.03880/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 9 Aug 2021.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2103.15595/) (2021)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/1908.02686/) (2019)
- [Deep Multi-View Stereo gone wild](https://mlsanity.com/paper/2104.15119/) (2021)
- [A Survey on Deep Learning Architectures for Image-based Depth Reconstruction](https://mlsanity.com/paper/1906.06113/) (2019)

## Built on by (most cited first, 7 of 7)

- [GeoNeRF: Generalizing NeRF with Geometry Priors](https://mlsanity.com/paper/2111.13539/) (2021)
- [RC-MVSNet: Unsupervised Multi-View Stereo with Neural Rendering](https://mlsanity.com/paper/2203.03949/) (2022)
- [CIRCLE: Convolutional Implicit Reconstruction and Completion for Large-scale Indoor Scene](https://mlsanity.com/paper/2111.12905/) (2021)
- [Domain-Adaptive Full-Face Gaze Estimation via Novel-View-Synthesis and Feature Disentanglement](https://mlsanity.com/paper/2305.16140/) (2023)
- [MultimodalStudio: A Heterogeneous Sensor Dataset and Framework for Neural Rendering across Multiple Imaging Modalities](https://mlsanity.com/paper/2503.19673/) (2025)
- [rpcPRF: Generalizable MPI Neural Radiance Field for Satellite Camera](https://mlsanity.com/paper/2310.07179/) (2023)
- [High Dynamic Range Novel View Synthesis with Single Exposure](https://mlsanity.com/paper/2505.01212/) (2025)

## Abstract (arXiv)

Multi-View Stereo (MVS) is a core task in 3D computer vision. With the surge of novel deep learning methods, learned MVS has surpassed the accuracy of classical approaches, but still relies on building a memory intensive dense cost volume. Novel View Synthesis (NVS) is a parallel line of research and has recently seen an increase in popularity with Neural Radiance Field (NeRF) models, which optimize a per scene radiance field. However, NeRF methods do not generalize to novel scenes and are slow to train and test. We propose to bridge the gap between these two methodologies with a novel network that can recover 3D scene geometry as a distance function, together with high-resolution color images. Our method uses only a sparse set of images as input and can generalize well to novel scenes. Additionally, we propose a coarse-to-fine sphere tracing approach in order to significantly increase speed. We show on various datasets that our method reaches comparable accuracy to per-scene optimized methods while being able to generalize and running significantly faster. We provide the source code at https://github.com/AIS-Bonn/neural_mvs