# OmniMVS: End-to-End Learning for Omnidirectional Stereo Matching

arXiv: https://arxiv.org/abs/1908.06257 · HTML: https://mlsanity.com/paper/1908.06257/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2019.

## Built on by (most cited first, 21 of 21)

- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey](https://mlsanity.com/paper/2406.19675/) (2024)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [3D Scene Geometry Estimation from 360^circ Imagery: A Survey](https://mlsanity.com/paper/2401.09252/) (2024)
- [PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimation](https://mlsanity.com/paper/2202.01323/) (2022)
- [Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision](https://mlsanity.com/paper/2302.09922/) (2023)
- [RomniStereo: Recurrent Omnidirectional Stereo Matching](https://mlsanity.com/paper/2401.04345/) (2024)
- [FoVA-Depth: Field-of-View Agnostic Depth Estimation for Cross-Dataset Generalization](https://mlsanity.com/paper/2401.13786/) (2024)
- [SDGE: Stereo Guided Depth Estimation for 360^circ Camera Sets](https://mlsanity.com/paper/2402.11791/) (2024)
- [Geometry-Informed Distance Candidate Selection for Adaptive Lightweight Omnidirectional Stereo Vision with Fisheye Images](https://mlsanity.com/paper/2405.05355/) (2024)
- [X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Cameras](https://mlsanity.com/paper/2607.12993/) (2026)
- [DaF-BEVSeg: Distortion-aware Fisheye Camera based Bird's Eye View Segmentation with Occlusion Reasoning](https://mlsanity.com/paper/2404.06352/) (2024)
- [MSI-NeRF: Linking Omni-Depth with View Synthesis through Multi-Sphere Image aided Generalizable Neural Radiance Field](https://mlsanity.com/paper/2403.10840/) (2024)
- [360 in the Wild: Dataset for Depth Prediction and View Synthesis](https://mlsanity.com/paper/2406.18898/) (2024)
- [Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model](https://mlsanity.com/paper/2503.23502/) (2025)
- [MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas](https://mlsanity.com/paper/2408.01653/) (2024)
- [Real-time Multi-view Omnidirectional Depth Estimation for Real Scenarios based on Teacher-Student Learning with Unlabeled Data](https://mlsanity.com/paper/2409.07843/) (2024)
- [Robust and Flexible Omnidirectional Depth Estimation with Multiple 360-degree Cameras](https://mlsanity.com/paper/2409.14766/) (2024)
- [Reference-Free Omnidirectional Stereo Matching via Multi-View Consistency Maximization](https://mlsanity.com/paper/2603.15019/) (2026)
- [OmniDS: Dual-Stream Context Fusion for Omnidirectional Depth from Fisheye Cameras](https://mlsanity.com/paper/2607.03038/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel end-to-end deep neural network model for omnidirectional depth estimation from a wide-baseline multi-view stereo setup. The images captured with ultra wide field-of-view (FOV) cameras on an omnidirectional rig are processed by the feature extraction module, and then the deep feature maps are warped onto the concentric spheres swept through all candidate depths using the calibrated camera parameters. The 3D encoder-decoder block takes the aligned feature volume to produce the omnidirectional depth estimate with regularization on uncertain regions utilizing the global context information. In addition, we present large-scale synthetic datasets for training and testing omnidirectional multi-view stereo algorithms. Our datasets consist of 11K ground-truth depth maps and 45K fisheye images in four orthogonal directions with various objects and environments. Experimental results show that the proposed method generates excellent results in both synthetic and real-world environments, and it outperforms the prior art and the omnidirectional versions of the state-of-the-art conventional stereo algorithms.