# 3D Scene Geometry Estimation from 360^circ Imagery: A Survey

arXiv: https://arxiv.org/abs/2401.09252 · HTML: https://mlsanity.com/paper/2401.09252/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 236 references that are in the map.
- First posted on arXiv 17 Jan 2024.

## Builds on (most cited first, 30 of 56)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Spherical CNNs](https://mlsanity.com/paper/1801.10130/) (2018)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering](https://mlsanity.com/paper/2106.02634/) (2021)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Learning SO(3) Equivariant Representations with Spherical CNNs](https://mlsanity.com/paper/1711.06721/) (2017)
- [A Survey of Structure from Motion](https://mlsanity.com/paper/1701.08493/) (2017)
- [HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation](https://mlsanity.com/paper/1901.03861/) (2019)
- [HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features](https://mlsanity.com/paper/2011.11498/) (2020)
- [OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas](https://mlsanity.com/paper/1807.09620/) (2018)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimation](https://mlsanity.com/paper/2102.03550/) (2021)
- [Spherical CNNs on Unstructured Grids](https://mlsanity.com/paper/1901.02039/) (2019)
- [Tangent Images for Mitigating Spherical Distortion](https://mlsanity.com/paper/1912.09390/) (2019)
- [DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama](https://mlsanity.com/paper/1811.11977/) (2018)
- [DeepSphere: a graph-based spherical CNN](https://mlsanity.com/paper/2012.15000/) (2020)

## Built on by (most cited first, 11 of 11)

- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [Conditional Panoramic Image Generation via Masked Autoregressive Modeling](https://mlsanity.com/paper/2505.16862/) (2025)
- [EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matching](https://mlsanity.com/paper/2502.20685/) (2025)
- [RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization](https://mlsanity.com/paper/2509.23991/) (2025)
- [Elite360M: Efficient 360 Multi-task Learning via Bi-projection Fusion and Cross-task Collaboration](https://mlsanity.com/paper/2408.09336/) (2024)
- [ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers](https://mlsanity.com/paper/2409.01216/) (2024)
- [Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling](https://mlsanity.com/paper/2606.27745/) (2026)
- [Deep Spherical Superpixels](https://mlsanity.com/paper/2407.17354/) (2024)
- [A Survey on Text-Driven 360-Degree Panorama Generation](https://mlsanity.com/paper/2502.14799/) (2025)
- [Generalized Shortest Path-based Superpixels for 3D Spherical Image Segmentation](https://mlsanity.com/paper/2509.19895/) (2025)
- [Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics](https://mlsanity.com/paper/2604.24642/) (2026)

## Abstract (arXiv)

This paper provides a comprehensive survey on pioneer and state-of-the-art 3D scene geometry estimation methodologies based on single, two, or multiple images captured under the omnidirectional optics. We first revisit the basic concepts of the spherical camera model, and review the most common acquisition technologies and representation formats suitable for omnidirectional (also called 360^circ, spherical or panoramic) images and videos. We then survey monocular layout and depth inference approaches, highlighting the recent advances in learning-based solutions suited for spherical data. The classical stereo matching is then revised on the spherical domain, where methodologies for detecting and describing sparse and dense features become crucial. The stereo matching concepts are then extrapolated for multiple view camera setups, categorizing them among light fields, multi-view stereo, and structure from motion (or visual simultaneous localization and mapping). We also compile and discuss commonly adopted datasets and figures of merit indicated for each purpose and list recent results for completeness. We conclude this paper by pointing out current and future trends.