# MVSTER: Epipolar Transformer for Efficient Multi-View Stereo

arXiv: https://arxiv.org/abs/2204.07346 · HTML: https://mlsanity.com/paper/2204.07346/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 80 references that are in the map.
- First posted on arXiv 15 Apr 2022.

## Builds on (most cited first, 30 of 57)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BERT Rediscovers the Classical NLP Pipeline](https://mlsanity.com/paper/1905.05950/) (2019)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)

## Built on by (most cited first, 30 of 50)

- [MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth](https://mlsanity.com/paper/2208.02541/) (2022)
- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Tracking Objects as Pixel-wise Distributions](https://mlsanity.com/paper/2207.05518/) (2022)
- [MVSAnywhere: Zero-Shot Multi-View Stereo](https://mlsanity.com/paper/2503.22430/) (2025)
- [Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes](https://mlsanity.com/paper/2304.08993/) (2023)
- [S-VolSDF: Sparse Multi-View Stereo Regularization of Neural Implicit Surfaces](https://mlsanity.com/paper/2303.17712/) (2023)
- [Multi-view Reconstruction via SfM-guided Monocular Depth Estimation](https://mlsanity.com/paper/2503.14483/) (2025)
- [C2F2NeUS: Cascade Cost Frustum Fusion for High Fidelity and Generalizable Neural Surface Reconstruction](https://mlsanity.com/paper/2306.10003/) (2023)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [When Epipolar Constraint Meets Non-local Operators in Multi-View Stereo](https://mlsanity.com/paper/2309.17218/) (2023)
- [Crafting Monocular Cues and Velocity Guidance for Self-Supervised Multi-Frame Depth Learning](https://mlsanity.com/paper/2208.09170/) (2022)
- [Two-View Geometry Scoring Without Correspondences](https://mlsanity.com/paper/2306.01596/) (2023)
- [SED-MVS: Segmentation-Driven and Edge-Aligned Deformation Multi-View Stereo with Depth Restoration and Occlusion Constraint](https://mlsanity.com/paper/2503.13721/) (2025)
- [INPC: Implicit Neural Point Clouds for Radiance Field Rendering](https://mlsanity.com/paper/2403.16862/) (2024)
- [CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo](https://mlsanity.com/paper/2305.10320/) (2023)
- [Learning Multi-view Anomaly Detection with Efficient Adaptive Selection](https://mlsanity.com/paper/2407.11935/) (2024)
- [Constraining Depth Map Geometry for Multi-View Stereo: A Dual-Depth Approach with Saddle-shaped Depth Cells](https://mlsanity.com/paper/2307.09160/) (2023)
- [Hierarchical Prior Mining for Non-local Multi-View Stereo](https://mlsanity.com/paper/2303.09758/) (2023)
- [Seeing World Dynamics in a Nutshell](https://mlsanity.com/paper/2502.03465/) (2025)
- [COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval](https://mlsanity.com/paper/2405.04103/) (2024)
- [RobustMVS: Single Domain Generalized Deep Multi-view Stereo](https://mlsanity.com/paper/2405.09131/) (2024)
- [Direct and Explicit 3D Generation from a Single Image](https://mlsanity.com/paper/2411.10947/) (2024)
- [Multi-Granularity Distillation Scheme Towards Lightweight Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.10169/) (2022)
- [RPBG: Towards Robust Neural Point-based Graphics in the Wild](https://mlsanity.com/paper/2405.05663/) (2024)
- [DVP-MVS++: Synergize Depth-Normal-Edge and Harmonized Visibility Prior for Multi-View Stereo](https://mlsanity.com/paper/2506.13215/) (2025)
- [Pixel-Aligned Multi-View Generation with Depth Guided Decoder](https://mlsanity.com/paper/2408.14016/) (2024)
- [CL-MVSNet: Unsupervised Multi-view Stereo with Dual-level Contrastive Learning](https://mlsanity.com/paper/2503.08219/) (2025)
- [Lightweight and Accurate Multi-View Stereo with Confidence-Aware Diffusion Model](https://mlsanity.com/paper/2509.15220/) (2025)
- [Look Around and Pay Attention: Multi-camera Point Tracking Reimagined with Transformers](https://mlsanity.com/paper/2512.04213/) (2025)
- [Laser: Latent Set Representations for 3D Generative Modeling](https://mlsanity.com/paper/2301.05747/) (2023)

## Abstract (arXiv)

Learning-based Multi-View Stereo (MVS) methods warp source images into the reference camera frustum to form 3D volumes, which are fused as a cost volume to be regularized by subsequent networks. The fusing step plays a vital role in bridging 2D semantics and 3D spatial associations. However, previous methods utilize extra networks to learn 2D information as fusing cues, underusing 3D spatial correlations and bringing additional computation costs. Therefore, we present MVSTER, which leverages the proposed epipolar Transformer to learn both 2D semantics and 3D spatial associations efficiently. Specifically, the epipolar Transformer utilizes a detachable monocular depth estimator to enhance 2D semantics and uses cross-attention to construct data-dependent 3D associations along epipolar line. Additionally, MVSTER is built in a cascade structure, where entropy-regularized optimal transport is leveraged to propagate finer depth estimations in each stage. Extensive experiments show MVSTER achieves state-of-the-art reconstruction performance with significantly higher efficiency: Compared with MVSNet and CasMVSNet, our MVSTER achieves 34% and 14% relative improvements on the DTU benchmark, with 80% and 51% relative reductions in running time. MVSTER also ranks first on Tanks&Temples-Advanced among all published works. Code is released at https://github.com/JeffWang987.