# 3DVNet: Multi-View Depth Prediction and Volumetric Refinement

arXiv: https://arxiv.org/abs/2112.00202 · HTML: https://mlsanity.com/paper/2112.00202/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 34 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinement](https://mlsanity.com/paper/2003.13017/) (2020)
- [DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2012.02177/) (2020)
- [MVDepthNet: Real-time Multiview Depth Estimation Neural Network](https://mlsanity.com/paper/1807.08563/) (2018)
- [Multi-View Stereo by Temporal Nonparametric Fusion](https://mlsanity.com/paper/1904.06397/) (2019)
- [Pyramid Multi-view Stereo Net with Self-adaptive View Aggregation](https://mlsanity.com/paper/1912.03001/) (2019)
- [MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions](https://mlsanity.com/paper/2104.13325/) (2021)

## Built on by (most cited first, 21 of 21)

- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors](https://mlsanity.com/paper/2209.15153/) (2022)
- [FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction](https://mlsanity.com/paper/2304.01480/) (2023)
- [CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction](https://mlsanity.com/paper/2304.14633/) (2023)
- [Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)](https://mlsanity.com/paper/2209.00082/) (2022)
- [DoubleTake: Geometry Guided Depth Estimation](https://mlsanity.com/paper/2406.18387/) (2024)
- [VisFusion: Visibility-aware Online 3D Scene Reconstruction from Videos](https://mlsanity.com/paper/2304.10687/) (2023)
- [MoGe-3: Fine-Detail Monocular Geometry Estimation with Self-Guided Sparse Volumetric Refinement](https://mlsanity.com/paper/2607.17967/) (2026)
- [NeuralRoom: Geometry-Constrained Neural Implicit Surfaces for Indoor Scene Reconstruction](https://mlsanity.com/paper/2210.06853/) (2022)
- [Unified Scene Representation and Reconstruction for 3D Large Language Models](https://mlsanity.com/paper/2404.13044/) (2024)
- [ProMotion: Prototypes As Motion Learners](https://mlsanity.com/paper/2406.04999/) (2024)
- [SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo](https://mlsanity.com/paper/2306.08648/) (2023)
- [Range-Agnostic Multi-View Depth Estimation With Keyframe Selection](https://mlsanity.com/paper/2401.14401/) (2024)
- [LivePose: Online 3D Reconstruction from Monocular Video with Dynamic Camera Poses](https://mlsanity.com/paper/2304.00054/) (2023)
- [Depth on Demand: Streaming Dense Depth from a Low Frame Rate Active Sensor](https://mlsanity.com/paper/2409.08277/) (2024)
- [Cross-Dimensional Refined Learning for Real-Time 3D Visual Perception from Monocular Video](https://mlsanity.com/paper/2303.09248/) (2023)
- [A survey on deep learning approaches for data integration in autonomous driving system](https://mlsanity.com/paper/2306.11740/) (2023)
- [VHS: High-Resolution Iterative Stereo Matching with Visual Hull Priors](https://mlsanity.com/paper/2406.02552/) (2024)
- [Geometry-guided Feature Learning and Fusion for Indoor Scene Reconstruction](https://mlsanity.com/paper/2408.15608/) (2024)
- [Prism: Semi-Supervised Multi-View Stereo with Monocular Structure Priors](https://mlsanity.com/paper/2412.05771/) (2024)

## Abstract (arXiv)

We present 3DVNet, a novel multi-view stereo (MVS) depth-prediction method that combines the advantages of previous depth-based and volumetric MVS approaches. Our key idea is the use of a 3D scene-modeling network that iteratively updates a set of coarse depth predictions, resulting in highly accurate predictions which agree on the underlying scene geometry. Unlike existing depth-prediction techniques, our method uses a volumetric 3D convolutional neural network (CNN) that operates in world space on all depth maps jointly. The network can therefore learn meaningful scene-level priors. Furthermore, unlike existing volumetric MVS techniques, our 3D CNN operates on a feature-augmented point cloud, allowing for effective aggregation of multi-view information and flexible iterative refinement of depth maps. Experimental results show our method exceeds state-of-the-art accuracy in both depth prediction and 3D reconstruction metrics on the ScanNet dataset, as well as a selection of scenes from the TUM-RGBD and ICL-NUIM datasets. This shows that our method is both effective and generalizes to new settings.