# MSP-MVS: Multi-Granularity Segmentation Prior Guided Multi-View Stereo

arXiv: https://arxiv.org/abs/2407.19323 · HTML: https://mlsanity.com/paper/2407.19323/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 138 references that are in the map.
- First posted on arXiv 27 Jul 2024.

## Builds on (most cited first, 24 of 24)

- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Semantic-SAM: Segment and Recognize Anything at Any Granularity](https://mlsanity.com/paper/2307.04767/) (2023)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [Multi-Scale Geometric Consistency Guided Multi-View Stereo](https://mlsanity.com/paper/1904.08103/) (2019)
- [Fast-MVSNet: Sparse-to-Dense Multi-View Stereo With Learned Propagation and Gauss-Newton Refinement](https://mlsanity.com/paper/2003.13017/) (2020)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network](https://mlsanity.com/paper/2108.03824/) (2021)
- [CRS-Diff: Controllable Remote Sensing Image Generation with Diffusion Model](https://mlsanity.com/paper/2403.11614/) (2024)
- [Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts](https://mlsanity.com/paper/2405.11613/) (2024)
- [TAPA-MVS: Textureless-Aware PAtchMatch Multi-View Stereo](https://mlsanity.com/paper/1903.10929/) (2019)
- [BIMCV-R: A Landmark Dataset for 3D CT Text-Image Retrieval](https://mlsanity.com/paper/2403.15992/) (2024)
- [SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM optimization](https://mlsanity.com/paper/2401.06385/) (2024)
- [Digging into contrastive learning for robust depth estimation with diffusion models](https://mlsanity.com/paper/2404.09831/) (2024)
- [Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation](https://mlsanity.com/paper/2308.09917/) (2023)
- [Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing](https://mlsanity.com/paper/2310.11346/) (2023)
- [Joint Depth Prediction and Semantic Segmentation with Multi-View SAM](https://mlsanity.com/paper/2311.00134/) (2023)
- [E-CRF: Embedded Conditional Random Field for Boundary-caused Class Weights Confusion in Semantic Segmentation](https://mlsanity.com/paper/2112.07106/) (2021)
- [TokenUnify: Scaling Up Autoregressive Pretraining for Neuron Segmentation](https://mlsanity.com/paper/2405.16847/) (2024)
- [Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generation](https://mlsanity.com/paper/2401.14257/) (2024)
- [XVTP3D: Cross-view Trajectory Prediction Using Shared 3D Queries for Autonomous Driving](https://mlsanity.com/paper/2308.08764/) (2023)
- [EVAN: Evolutional Video Streaming Adaptation via Neural Representation](https://mlsanity.com/paper/2406.02557/) (2024)
- [StreamOptix: A Cross-layer Adaptive Video Delivery Scheme](https://mlsanity.com/paper/2406.04632/) (2024)

## Built on by (most cited first, 17 of 17)

- [Jasmine: Harnessing Diffusion Prior for Self-supervised Depth Estimation](https://mlsanity.com/paper/2503.15905/) (2025)
- [Layout2Scene: 3D Semantic Layout Guided Scene Generation via Geometry and Appearance Diffusion Priors](https://mlsanity.com/paper/2501.02519/) (2025)
- [Physical Autoregressive Model for Robotic Manipulation without Action Pretraining](https://mlsanity.com/paper/2508.09822/) (2025)
- [HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene](https://mlsanity.com/paper/2506.09518/) (2025)
- [Ultra-High Resolution Segmentation via Boundary-Enhanced Patch-Merging Transformer](https://mlsanity.com/paper/2412.10181/) (2024)
- [Light4GS: Lightweight Compact 4D Gaussian Splatting Generation via Context Model](https://mlsanity.com/paper/2503.13948/) (2025)
- [CoC-VLA: Delving into Adversarial Domain Transfer for Explainable Autonomous Driving via Chain-of-Causality Visual-Language-Action Model](https://mlsanity.com/paper/2511.19914/) (2025)
- [DVP-MVS++: Synergize Depth-Normal-Edge and Harmonized Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2506.13215/) (2025)
- [MonoMVSNet: Monocular Priors Guided Multi-View Stereo Network](https://mlsanity.com/paper/2507.11333/) (2025)
- [Efficient Feature Fusion for UAV Object Detection](https://mlsanity.com/paper/2501.17983/) (2025)
- [MVSMamba: Multi-View Stereo with State Space Model](https://mlsanity.com/paper/2511.01315/) (2025)
- [Dual form Complementary Masking for Domain-Adaptive Image Segmentation](https://mlsanity.com/paper/2507.12008/) (2025)
- [From Tokens to Nodes: Semantic-Guided Motion Control for Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2510.02732/) (2025)
- [LiMT: A Multi-task Liver Image Benchmark Dataset](https://mlsanity.com/paper/2511.19889/) (2025)
- [ZeroDiff: Zero-Shot Time Series Reconstruction via Informed-Prior Diffusion](https://mlsanity.com/paper/2609.37078/) (2026)
- [ZipMVS: Multi-View Stereo with Compressed Cost Volumes](https://mlsanity.com/paper/2608.28033/) (2026)
- [X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning](https://mlsanity.com/paper/2610.16811/) (2026)

## Abstract (arXiv)

Recently, patch deformation-based methods have demonstrated significant strength in multi-view stereo by adaptively expanding the reception field of patches to help reconstruct textureless areas. However, such methods mainly concentrate on searching for pixels without matching ambiguity (i.e., reliable pixels) when constructing deformed patches, while neglecting the deformation instability caused by unexpected edge-skipping, resulting in potential matching distortions. Addressing this, we propose MSP-MVS, a method introducing multi-granularity segmentation prior for edge-confined patch deformation. Specifically, to avoid unexpected edge-skipping, we first aggregate and further refine multi-granularity depth edges gained from Semantic-SAM as prior to guide patch deformation within depth-continuous (i.e., homogeneous) areas. Moreover, to address attention imbalance caused by edge-confined patch deformation, we implement adaptive equidistribution and disassemble-clustering of correlative reliable pixels (i.e., anchors), thereby promoting attention-consistent patch deformation. Finally, to prevent deformed patches from falling into local-minimum matching costs caused by the fixed sampling pattern, we introduce disparity-sampling synergistic 3D optimization to help identify global-minimum matching costs. Evaluations on ETH3D and Tanks & Temples benchmarks prove our method obtains state-of-the-art performance with remarkable generalization.