# Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

arXiv: https://arxiv.org/abs/2302.09922 · HTML: https://mlsanity.com/paper/2302.09922/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videos](https://mlsanity.com/paper/1806.01320/) (2018)
- [Anytime Stereo Image Depth Estimation on Mobile Devices](https://mlsanity.com/paper/1810.11408/) (2018)
- [Spherical View Synthesis for Self-Supervised 360 Depth Estimation](https://mlsanity.com/paper/1909.08112/) (2019)
- [SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2008.04017/) (2020)
- [LEGO: Learning Edge with Geometry all at Once by Watching Videos](https://mlsanity.com/paper/1803.05648/) (2018)
- [Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistency](https://mlsanity.com/paper/1711.03665/) (2017)
- [SweepNet: Wide-baseline Omnidirectional Depth Estimation](https://mlsanity.com/paper/1902.10904/) (2019)
- [OmniMVS: End-to-End Learning for Omnidirectional Stereo Matching](https://mlsanity.com/paper/1908.06257/) (2019)
- [360^circ Depth Estimation from Multiple Fisheye Images with Origami Crown Representation of Icosahedron](https://mlsanity.com/paper/2007.06891/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [MVG-Splatting: Multi-View Guided Gaussian Splatting with Adaptive Quantile-Based Geometric Consistency Densification](https://mlsanity.com/paper/2407.11840/) (2024)
- [Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimation](https://mlsanity.com/paper/2411.18335/) (2024)
- [Geometry-Informed Distance Candidate Selection for Adaptive Lightweight Omnidirectional Stereo Vision with Fisheye Images](https://mlsanity.com/paper/2405.05355/) (2024)
- [X-Lens: Real-Time Metric Depth Estimation with Heterogeneous Cameras](https://mlsanity.com/paper/2607.12993/) (2026)
- [MSI-NeRF: Linking Omni-Depth with View Synthesis through Multi-Sphere Image aided Generalizable Neural Radiance Field](https://mlsanity.com/paper/2403.10840/) (2024)
- [Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model](https://mlsanity.com/paper/2503.23502/) (2025)
- [Real-time Multi-view Omnidirectional Depth Estimation for Real Scenarios based on Teacher-Student Learning with Unlabeled Data](https://mlsanity.com/paper/2409.07843/) (2024)

## Abstract (arXiv)

Omnidirectional multi-view stereo (MVS) vision is attractive for its ultra-wide field-of-view (FoV), enabling machines to perceive 360deg 3D surroundings. However, the existing solutions require expensive dense depth labels for supervision, making them impractical in real-world applications. In this paper, we propose the first unsupervised omnidirectional MVS framework based on multiple fisheye images. To this end, we project all images to a virtual view center and composite two panoramic images with spherical geometry from two pairs of back-to-back fisheye images. The two 360deg images formulate a stereo pair with a special pose, and the photometric consistency is leveraged to establish the unsupervised constraint, which we term "Pseudo-Stereo Supervision". In addition, we propose Un-OmniMVS, an efficient unsupervised omnidirectional MVS network, to facilitate the inference speed with two efficient components. First, a novel feature extractor with frequency attention is proposed to simultaneously capture the non-local Fourier features and local spatial features, explicitly facilitating the feature representation. Then, a variance-based light cost volume is put forward to reduce the computational complexity. Experiments exhibit that the performance of our unsupervised solution is competitive to that of the state-of-the-art (SoTA) supervised methods with better generalization in real-world data.