# A Benchmark and a Baseline for Robust Multi-view Depth Estimation

arXiv: https://arxiv.org/abs/2209.06681 · HTML: https://mlsanity.com/paper/2209.06681/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [DeepV2D: Video to Depth with Differentiable Structure from Motion](https://mlsanity.com/paper/1812.04605/) (2018)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [Uncertainty Estimates and Multi-Hypotheses Networks for Optical Flow](https://mlsanity.com/paper/1802.07095/) (2018)
- [Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinement](https://mlsanity.com/paper/2003.13017/) (2020)
- [Visibility-aware Multi-view Stereo Network](https://mlsanity.com/paper/2008.07928/) (2020)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera](https://mlsanity.com/paper/1901.02571/) (2019)
- [MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions](https://mlsanity.com/paper/2104.13325/) (2021)
- [PVSNet: Pixelwise Visibility-Aware Multi-View Stereo Network](https://mlsanity.com/paper/2007.07714/) (2020)

## Built on by (most cited first, 30 of 39)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [MapAnything: Universal Feed-Forward Metric 3D Reconstruction](https://mlsanity.com/paper/2509.13414/) (2025)
- [MUSt3R: Multi-view Network for Stereo 3D Reconstruction](https://mlsanity.com/paper/2503.01661/) (2025)
- [Large Spatial Model: End-to-end Unposed Images to Semantic 3D](https://mlsanity.com/paper/2410.18956/) (2024)
- [Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors](https://mlsanity.com/paper/2503.17316/) (2025)
- [InstantSplat: Sparse-view Gaussian Splatting in Seconds](https://mlsanity.com/paper/2403.20309/) (2024)
- [World-consistent Video Diffusion with Explicit 3D Modeling](https://mlsanity.com/paper/2412.01821/) (2024)
- [RoMa v2: Harder Better Faster Denser Feature Matching](https://mlsanity.com/paper/2511.15706/) (2025)
- [Dens3R: A Foundation Model for 3D Geometry Prediction](https://mlsanity.com/paper/2507.16290/) (2025)
- [Matrix3D: Large Photogrammetry Model All-in-One](https://mlsanity.com/paper/2502.07685/) (2025)
- [UFM: A Simple Path towards Unified Dense Correspondence with Flow](https://mlsanity.com/paper/2506.09278/) (2025)
- [DUNE: Distilling a Universal Encoder from Heterogeneous 2D and 3D Teachers](https://mlsanity.com/paper/2503.14405/) (2025)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training](https://mlsanity.com/paper/2512.10950/) (2025)
- [Test3R: Learning to Reconstruct 3D at Test Time](https://mlsanity.com/paper/2506.13750/) (2025)
- [PanSt3R: Multi-view Consistent Panoptic Segmentation](https://mlsanity.com/paper/2506.21348/) (2025)
- [AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend](https://mlsanity.com/paper/2511.20343/) (2025)
- [PE3R: Perception-Efficient 3D Reconstruction](https://mlsanity.com/paper/2503.07507/) (2025)
- [Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better](https://mlsanity.com/paper/2503.19904/) (2025)
- [Regist3R: Incremental Registration with Stereo Foundation Model](https://mlsanity.com/paper/2504.12356/) (2025)
- [Flow3r: Factored Flow Prediction for Scalable Visual Geometry Learning](https://mlsanity.com/paper/2602.20157/) (2026)
- [LagerNVS: Latent Geometry for Fully Neural Real-time Novel View Synthesis](https://mlsanity.com/paper/2603.20176/) (2026)
- [Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow](https://mlsanity.com/paper/2602.14021/) (2026)
- [Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation](https://mlsanity.com/paper/2511.22429/) (2025)
- [Range-Agnostic Multi-View Depth Estimation With Keyframe Selection](https://mlsanity.com/paper/2401.14401/) (2024)
- [SceneFactory: A Workflow-centric and Unified Framework for Incremental Scene Modeling](https://mlsanity.com/paper/2405.07847/) (2024)
- [MoRe: Monocular Geometry Refinement via Graph Optimization for Cross-View Consistency](https://mlsanity.com/paper/2510.07119/) (2025)
- [MonoPatchNeRF: Improving Neural Radiance Fields with Patch-based Monocular Guidance](https://mlsanity.com/paper/2404.08252/) (2024)
- [UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perception](https://mlsanity.com/paper/2602.23224/) (2026)
- [SimpleProc: Fully Procedural Synthetic Data from Simple Rules for Multi-View Stereo](https://mlsanity.com/paper/2604.04925/) (2026)

## Abstract (arXiv)

Recent deep learning approaches for multi-view depth estimation are employed either in a depth-from-video or a multi-view stereo setting. Despite different settings, these approaches are technically similar: they correlate multiple source views with a keyview to estimate a depth map for the keyview. In this work, we introduce the Robust Multi-View Depth Benchmark that is built upon a set of public datasets and allows evaluation in both settings on data from different domains. We evaluate recent approaches and find imbalanced performances across domains. Further, we consider a third setting, where camera poses are available and the objective is to estimate the corresponding depth maps with their correct scale. We show that recent approaches do not generalize across datasets in this setting. This is because their cost volume output runs out of distribution. To resolve this, we present the Robust MVD Baseline model for multi-view depth estimation, which is built upon existing components but employs a novel scale augmentation procedure. It can be applied for robust multi-view depth estimation, independent of the target data. We provide code for the proposed benchmark and baseline model at https://github.com/lmb-freiburg/robustmvd.