# SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM optimization

arXiv: https://arxiv.org/abs/2401.06385 · HTML: https://mlsanity.com/paper/2401.06385/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 111 references that are in the map.
- First posted on arXiv 12 Jan 2024.

## Builds on (most cited first, 10 of 10)

- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Multi-Scale Geometric Consistency Guided Multi-View Stereo](https://mlsanity.com/paper/1904.08103/) (2019)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network](https://mlsanity.com/paper/2108.03824/) (2021)
- [Planar Prior Assisted PatchMatch Multi-View Stereo](https://mlsanity.com/paper/1912.11744/) (2019)
- [TAPA-MVS: Textureless-Aware PAtchMatch Multi-View Stereo](https://mlsanity.com/paper/1903.10929/) (2019)
- [PatchMatch-RL: Deep MVS with Pixelwise Depth, Normal, and Visibility](https://mlsanity.com/paper/2108.08943/) (2021)
- [XVTP3D: Cross-view Trajectory Prediction Using Shared 3D Queries for Autonomous Driving](https://mlsanity.com/paper/2308.08764/) (2023)

## Built on by (most cited first, 19 of 19)

- [GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting](https://mlsanity.com/paper/2408.11447/) (2024)
- [OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval](https://mlsanity.com/paper/2507.05631/) (2025)
- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View Stereo](https://mlsanity.com/paper/2407.19323/) (2024)
- [DVP-MVS: Synergize Depth-Edge and Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2412.11578/) (2024)
- [Physical Autoregressive Model for Robotic Manipulation without Action Pretraining](https://mlsanity.com/paper/2508.09822/) (2025)
- [Conditional Latent Coding with Learnable Synthesized Reference for Deep Image Compression](https://mlsanity.com/paper/2502.09971/) (2025)
- [SED-MVS: Segmentation-Driven and Edge-Aligned Deformation Multi-View Stereo with Depth Restoration and Occlusion Constraint](https://mlsanity.com/paper/2503.13721/) (2025)
- [HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene](https://mlsanity.com/paper/2506.09518/) (2025)
- [SIRI-Bench: Challenging VLMs' Spatial Intelligence through Complex Reasoning Tasks](https://mlsanity.com/paper/2506.14512/) (2025)
- [Dual-Level Precision Edges Guided Multi-View Stereo with Accurate Planarization](https://mlsanity.com/paper/2412.20328/) (2024)
- [DVP-MVS++: Synergize Depth-Normal-Edge and Harmonized Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2506.13215/) (2025)
- [Human-Centric Open-Future Task Discovery: Formulation, Benchmark, and Scalable Tree-Based Search](https://mlsanity.com/paper/2511.18929/) (2025)
- [Aerial Multi-View Stereo via Adaptive Depth Range Inference and Normal Cues](https://mlsanity.com/paper/2506.05655/) (2025)
- [Dual form Complementary Masking for Domain-Adaptive Image Segmentation](https://mlsanity.com/paper/2507.12008/) (2025)
- [From Tokens to Nodes: Semantic-Guided Motion Control for Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2510.02732/) (2025)
- [LiMT: A Multi-task Liver Image Benchmark Dataset](https://mlsanity.com/paper/2511.19889/) (2025)
- [Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction](https://mlsanity.com/paper/2605.11354/) (2026)
- [X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning](https://mlsanity.com/paper/2610.16811/) (2026)

## Abstract (arXiv)

In this paper, we introduce Segmentation-Driven Deformation Multi-View Stereo (SD-MVS), a method that can effectively tackle challenges in 3D reconstruction of textureless areas. We are the first to adopt the Segment Anything Model (SAM) to distinguish semantic instances in scenes and further leverage these constraints for pixelwise patch deformation on both matching cost and propagation. Concurrently, we propose a unique refinement strategy that combines spherical coordinates and gradient descent on normals and pixelwise search interval on depths, significantly improving the completeness of reconstructed 3D model. Furthermore, we adopt the Expectation-Maximization (EM) algorithm to alternately optimize the aggregate matching cost and hyperparameters, effectively mitigating the problem of parameters being excessively dependent on empirical tuning. Evaluations on the ETH3D high-resolution multi-view stereo benchmark and the Tanks and Temples dataset demonstrate that our method can achieve state-of-the-art results with less time consumption.