# Constraining Depth Map Geometry for Multi-View Stereo: A Dual-Depth Approach with Saddle-shaped Depth Cells

arXiv: https://arxiv.org/abs/2307.09160 · HTML: https://mlsanity.com/paper/2307.09160/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 18 Jul 2023.

## Builds on (most cited first, 26 of 26)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [Point-Based Multi-View Stereo Network](https://mlsanity.com/paper/1908.04422/) (2019)
- [SurfaceNet: An End-to-end 3D Neural Network for Multiview Stereopsis](https://mlsanity.com/paper/1708.01749/) (2017)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [StereoNet: Guided Hierarchical Refinement for Real-Time Edge-Aware Depth Prediction](https://mlsanity.com/paper/1807.08865/) (2018)
- [Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation](https://mlsanity.com/paper/2201.01501/) (2022)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network](https://mlsanity.com/paper/2108.03824/) (2021)
- [MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth](https://mlsanity.com/paper/2208.02541/) (2022)
- [MVSTER: Epipolar Transformer for Efficient Multi-View Stereo](https://mlsanity.com/paper/2204.07346/) (2022)
- [Multi-View Fusion of Sensor Data for Improved Perception and Prediction in Autonomous Driving](https://mlsanity.com/paper/2008.11901/) (2020)
- [Less is More: Consistent Video Depth Estimation with Masked Frames Modeling](https://mlsanity.com/paper/2208.00380/) (2022)
- [Learning Second-Order Attentive Context for Efficient Correspondence Pruning](https://mlsanity.com/paper/2303.15761/) (2023)
- [A2B: Anchor to Barycentric Coordinate for Robust Correspondence](https://mlsanity.com/paper/2306.02760/) (2023)
- [Learning Probabilistic Coordinate Fields for Robust Correspondences](https://mlsanity.com/paper/2306.04231/) (2023)

## Built on by (most cited first, 8 of 8)

- [DUSt3R: Geometric 3D Vision Made Easy](https://mlsanity.com/paper/2312.14132/) (2023)
- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [JOG3R: Towards 3D-Consistent Video Generators](https://mlsanity.com/paper/2501.01409/) (2025)
- [MonoMVSNet: Monocular Priors Guided Multi-View Stereo Network](https://mlsanity.com/paper/2507.11333/) (2025)
- [MVSMamba: Multi-View Stereo with State Space Model](https://mlsanity.com/paper/2511.01315/) (2025)
- [MuGS: Multi-Baseline Generalizable Gaussian Splatting Reconstruction](https://mlsanity.com/paper/2508.04297/) (2025)
- [3DReflecNet: A Large-Scale Dataset for 3D Reconstruction of Reflective, Transparent, and Low-Texture Objects](https://mlsanity.com/paper/2605.10204/) (2026)

## Abstract (arXiv)

Learning-based multi-view stereo (MVS) methods deal with predicting accurate depth maps to achieve an accurate and complete 3D representation. Despite the excellent performance, existing methods ignore the fact that a suitable depth geometry is also critical in MVS. In this paper, we demonstrate that different depth geometries have significant performance gaps, even using the same depth prediction error. Therefore, we introduce an ideal depth geometry composed of Saddle-Shaped Cells, whose predicted depth map oscillates upward and downward around the ground-truth surface, rather than maintaining a continuous and smooth depth plane. To achieve it, we develop a coarse-to-fine framework called Dual-MVSNet (DMVSNet), which can produce an oscillating depth plane. Technically, we predict two depth values for each pixel (Dual-Depth), and propose a novel loss function and a checkerboard-shaped selecting strategy to constrain the predicted depth geometry. Compared to existing methods,DMVSNet achieves a high rank on the DTU benchmark and obtains the top performance on challenging scenes of Tanks and Temples, demonstrating its strong performance and generalization ability. Our method also points to a new research direction for considering depth geometry in MVS.