# MVDepthNet: Real-time Multiview Depth Estimation Neural Network

arXiv: https://arxiv.org/abs/1807.08563 · HTML: https://mlsanity.com/paper/1807.08563/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 30 of 59)

- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereo](https://mlsanity.com/paper/2109.01129/) (2021)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Seconds](https://mlsanity.com/paper/2412.06974/) (2024)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [VolumeFusion: Deep Depth Fusion for 3D Scene Reconstruction](https://mlsanity.com/paper/2108.08623/) (2021)
- [VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion](https://mlsanity.com/paper/2112.00236/) (2021)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [When the Sun Goes Down: Repairing Photometric Losses for All-Day Depth Estimation](https://mlsanity.com/paper/2206.13850/) (2022)
- [3DVNet: Multi-View Depth Prediction and Volumetric Refinement](https://mlsanity.com/paper/2112.00202/) (2021)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [Exploring Geometric Consistency for Monocular 3D Object Detection](https://mlsanity.com/paper/2104.05858/) (2021)
- [MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors](https://mlsanity.com/paper/2209.15153/) (2022)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction](https://mlsanity.com/paper/2304.14633/) (2023)
- [Two-level Data Augmentation for Calibrated Multi-view Detection](https://mlsanity.com/paper/2210.10756/) (2022)
- [One at a Time: Progressive Multi-step Volumetric Probability Learning for Reliable 3D Scene Perception](https://mlsanity.com/paper/2306.12681/) (2023)
- [A Confidence-based Iterative Solver of Depths and Surface Normals for Deep Multi-view Stereo](https://mlsanity.com/paper/2201.07609/) (2022)
- [RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo](https://mlsanity.com/paper/2205.14320/) (2022)
- [DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking](https://mlsanity.com/paper/2303.16628/) (2023)
- [MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Pose](https://mlsanity.com/paper/2210.07181/) (2022)
- [DoubleTake: Geometry Guided Depth Estimation](https://mlsanity.com/paper/2406.18387/) (2024)
- [3D Former: Monocular Scene Reconstruction with 3D SDF Transformers](https://mlsanity.com/paper/2301.13510/) (2023)
- [VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos](https://mlsanity.com/paper/2304.10687/) (2023)
- [ID-Pose: Sparse-view Camera Pose Estimation by Inverting Diffusion Models](https://mlsanity.com/paper/2306.17140/) (2023)
- [Deep Depth from Focal Stack with Defocus Model for Camera-Setting Invariance](https://mlsanity.com/paper/2202.13055/) (2022)
- [StandardSim: A Synthetic Dataset For Retail Environments](https://mlsanity.com/paper/2202.02418/) (2022)

## Abstract (arXiv)

Although deep neural networks have been widely applied to computer vision problems, extending them into multiview depth estimation is non-trivial. In this paper, we present MVDepthNet, a convolutional network to solve the depth estimation problem given several image-pose pairs from a localized monocular camera in neighbor viewpoints. Multiview observations are encoded in a cost volume and then combined with the reference image to estimate the depth map using an encoder-decoder network. By encoding the information from multiview observations into the cost volume, our method achieves real-time performance and the flexibility of traditional methods that can be applied regardless of the camera intrinsic parameters and the number of images. Geometric data augmentation is used to train MVDepthNet. We further apply MVDepthNet in a monocular dense mapping system that continuously estimates depth maps using a single localized moving camera. Experiments show that our method can generate depth maps efficiently and precisely.