# WT-MVSNet: Window-based Transformers for Multi-view Stereo

arXiv: https://arxiv.org/abs/2205.14319 · HTML: https://mlsanity.com/paper/2205.14319/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 37 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networks](https://mlsanity.com/paper/1911.10127/) (2019)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inference](https://mlsanity.com/paper/1902.10556/) (2019)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Cost Volume Pyramid Based Depth Inference for Multi-View Stereo](https://mlsanity.com/paper/1912.08329/) (2019)
- [Deep Stereo using Adaptive Thin Volume Representation with Uncertainty Awareness](https://mlsanity.com/paper/1911.12012/) (2019)
- [DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2106.06716/) (2021)
- [TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers](https://mlsanity.com/paper/2111.14600/) (2021)
- [Multi-Scale Geometric Consistency Guided Multi-View Stereo](https://mlsanity.com/paper/1904.08103/) (2019)
- [Visibility-aware Multi-view Stereo Network](https://mlsanity.com/paper/2008.07928/) (2020)
- [Rethinking Depth Estimation for Multi-View Stereo: A Unified Representation](https://mlsanity.com/paper/2201.01501/) (2022)
- [Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checking](https://mlsanity.com/paper/2007.10872/) (2020)
- [AA-RMVSNet: Adaptive Aggregation Recurrent Multi-view Stereo Network](https://mlsanity.com/paper/2108.03824/) (2021)
- [DeepVideoMVS: Multi-View Stereo on Video with Recurrent Spatio-Temporal Fusion](https://mlsanity.com/paper/2012.02177/) (2020)
- [IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereo](https://mlsanity.com/paper/2112.05126/) (2021)
- [Pyramid Multi-view Stereo Net with Self-adaptive View Aggregation](https://mlsanity.com/paper/1912.03001/) (2019)

## Built on by (most cited first, 20 of 20)

- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion](https://mlsanity.com/paper/2303.13959/) (2023)
- [MonSter++: Unified Stereo Matching, Multi-view Stereo, and Real-time Stereo with Monodepth Priors](https://mlsanity.com/paper/2501.08643/) (2025)
- [KD-MVS: Knowledge Distillation Based Self-supervised Learning for Multi-view Stereo](https://mlsanity.com/paper/2207.10425/) (2022)
- [C2F2NeUS: Cascade Cost Frustum Fusion for High Fidelity and Generalizable Neural Surface Reconstruction](https://mlsanity.com/paper/2306.10003/) (2023)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [CostFormer:Cost Transformer for Cost Aggregation in Multi-view Stereo](https://mlsanity.com/paper/2305.10320/) (2023)
- [Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction](https://mlsanity.com/paper/2412.08243/) (2024)
- [XFMamba: Cross-Fusion Mamba for Multi-View Medical Image Classification](https://mlsanity.com/paper/2503.02619/) (2025)
- [MonoMVSNet: Monocular Priors Guided Multi-View Stereo Network](https://mlsanity.com/paper/2507.11333/) (2025)
- [ARAI-MVSNet: A multi-view stereo depth estimation network with adaptive depth range and depth interval](https://mlsanity.com/paper/2308.09022/) (2023)
- [CT-MVSNet: Efficient Multi-View Stereo with Cross-scale Transformer](https://mlsanity.com/paper/2312.08594/) (2023)
- [MVSMamba: Multi-View Stereo with State Space Model](https://mlsanity.com/paper/2511.01315/) (2025)
- [Transparent Object Depth Completion](https://mlsanity.com/paper/2405.15299/) (2024)
- [Refinement of Monocular Depth Maps via Multi-View Differentiable Rendering](https://mlsanity.com/paper/2410.03861/) (2024)
- [Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction](https://mlsanity.com/paper/2605.11354/) (2026)
- [Prism: Semi-Supervised Multi-View Stereo with Monocular Structure Priors](https://mlsanity.com/paper/2412.05771/) (2024)
- [Detail-aware multi-view stereo network for depth estimation](https://mlsanity.com/paper/2503.23684/) (2025)
- [IA-MVS: Instance-Focused Adaptive Depth Sampling for Multi-View Stereo](https://mlsanity.com/paper/2505.12714/) (2025)
- [M2Depth: Unifying Monocular Depth Foundation Priors with Multi-View Stereo](https://mlsanity.com/paper/2608.20788/) (2026)

## Abstract (arXiv)

Recently, Transformers were shown to enhance the performance of multi-view stereo by enabling long-range feature interaction. In this work, we propose Window-based Transformers (WT) for local feature matching and global feature aggregation in multi-view stereo. We introduce a Window-based Epipolar Transformer (WET) which reduces matching redundancy by using epipolar constraints. Since point-to-line matching is sensitive to erroneous camera pose and calibration, we match windows near the epipolar lines. A second Shifted WT is employed for aggregating global information within cost volume. We present a novel Cost Transformer (CT) to replace 3D convolutions for cost volume regularization. In order to better constrain the estimated depth maps from multiple views, we further design a novel geometric consistency loss (Geo Loss) which punishes unreliable areas where multi-view consistency is not satisfied. Our WT multi-view stereo method (WT-MVSNet) achieves state-of-the-art performance across multiple datasets and ranks 1st on Tanks and Temples benchmark.