# Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks

arXiv: https://arxiv.org/abs/2011.13118 · HTML: https://mlsanity.com/paper/2011.13118/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2020.

## Built on by (most cited first, 30 of 59)

- [GaussianPro: 3D Gaussian Splatting with Progressive Propagation](https://mlsanity.com/paper/2402.14650/) (2024)
- [SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Views](https://mlsanity.com/paper/2206.05737/) (2022)
- [NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priors](https://mlsanity.com/paper/2206.13597/) (2022)
- [A Survey of Visual Transformers](https://mlsanity.com/paper/2111.06091/) (2021)
- [TransformerFusion: Monocular RGB Scene Reconstruction using Transformers](https://mlsanity.com/paper/2107.02191/) (2021)
- [Learning Temporally Consistent Video Depth from Video Diffusion Priors](https://mlsanity.com/paper/2406.01493/) (2024)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologies](https://mlsanity.com/paper/2211.14173/) (2022)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [Self-supervised Monocular Depth Estimation for All Day Images using Domain Separation](https://mlsanity.com/paper/2108.07628/) (2021)
- [MVS2D: Efficient Multi-view Stereo via Attention-Driven 2D Convolutions](https://mlsanity.com/paper/2104.13325/) (2021)
- [VoRTX: Volumetric 3D Reconstruction With Transformers for Voxelwise View Selection and Fusion](https://mlsanity.com/paper/2112.00236/) (2021)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)
- [OmniNWM: Omniscient Driving Navigation World Models](https://mlsanity.com/paper/2510.18313/) (2025)
- [Hierarchical Temporal Context Learning for Camera-based Semantic Scene Completion](https://mlsanity.com/paper/2407.02077/) (2024)
- [Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving](https://mlsanity.com/paper/2403.07535/) (2024)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2206.07710/) (2022)
- [MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation](https://mlsanity.com/paper/2307.14336/) (2023)
- [UC-NeRF: Neural Radiance Field for Under-Calibrated Multi-view Cameras in Autonomous Driving](https://mlsanity.com/paper/2311.16945/) (2023)
- [MonSter++: Unified Stereo Matching, Multi-view Stereo, and Real-time Stereo with Monodepth Priors](https://mlsanity.com/paper/2501.08643/) (2025)
- [Learning-based Multi-View Stereo: A Survey](https://mlsanity.com/paper/2408.15235/) (2024)
- [Depth Field Networks for Generalizable Multi-view Scene Representation](https://mlsanity.com/paper/2207.14287/) (2022)
- [Real-Time Dense 3D Mapping of Underwater Environments](https://mlsanity.com/paper/2304.02704/) (2023)
- [CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction](https://mlsanity.com/paper/2304.14633/) (2023)
- [MVTrans: Multi-View Perception of Transparent Objects](https://mlsanity.com/paper/2302.11683/) (2023)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)
- [One at a Time: Progressive Multi-step Volumetric Probability Learning for Reliable 3D Scene Perception](https://mlsanity.com/paper/2306.12681/) (2023)
- [FutureDepth: Learning to Predict the Future Improves Video Depth Estimation](https://mlsanity.com/paper/2403.12953/) (2024)
- [RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo](https://mlsanity.com/paper/2205.14320/) (2022)

## Abstract (arXiv)

We present a novel method for multi-view depth estimation from a single video, which is a critical task in various applications, such as perception, reconstruction and robot navigation. Although previous learning-based methods have demonstrated compelling results, most works estimate depth maps of individual video frames independently, without taking into consideration the strong geometric and temporal coherence among the frames. Moreover, current state-of-the-art (SOTA) models mostly adopt a fully 3D convolution network for cost regularization and therefore require high computational cost, thus limiting their deployment in real-world applications. Our method achieves temporally coherent depth estimation results by using a novel Epipolar Spatio-Temporal (EST) transformer to explicitly associate geometric and temporal correlation with multiple estimated depth maps. Furthermore, to reduce the computational cost, inspired by recent Mixture-of-Experts models, we design a compact hybrid network consisting of a 2D context-aware network and a 3D matching network which learn 2D context information and 3D disparity cues separately. Extensive experiments demonstrate that our method achieves higher accuracy in depth estimation and significant speedup than the SOTA methods.