mlsanityData to 7 Oct 2026

3D Scene Geometry Estimation from 360^circ Imagery: A Survey

arXiv 2401.09252 ↗First posted 17 Jan 2024cs.CV cs.AI cs.GRcited by 11

Not on the map. Why?

Abstract (arXiv)

This paper provides a comprehensive survey on pioneer and state-of-the-art 3D scene geometry estimation methodologies based on single, two, or multiple images captured under the omnidirectional optics. We first revisit the basic concepts of the spherical camera model, and review the most common acquisition technologies and representation formats suitable for omnidirectional (also called 360^circ, spherical or panoramic) images and videos. We then survey monocular layout and depth inference approaches, highlighting the recent advances in learning-based solutions suited for spherical data. The classical stereo matching is then revised on the spherical domain, where methodologies for detecting and describing sparse and dense features become crucial. The stereo matching concepts are then extrapolated for multiple view camera setups, categorizing them among light fields, multi-view stereo, and structure from motion (or visual simultaneous localization and mapping). We also compile and discuss commonly adopted datasets and figures of merit indicated for each purpose and list recent results for completeness. We conclude this paper by pointing out current and future trends.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 56 of its 236 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

56 of 236 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2017Deformable Convolutional Networkscited by 1,481
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  6. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  7. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  8. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  9. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  10. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  11. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  12. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
38 more
  1. 2018Spherical CNNscited by 452
  2. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  3. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  4. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  5. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 330
  6. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  7. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  8. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 244
  9. 2017A Survey of Structure from Motioncited by 157
  10. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 107
  11. 2020HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Featurescited by 104
  12. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 98
  13. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 94
  14. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  15. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  16. 2019Spherical CNNs on Unstructured Gridscited by 76
  17. 2018DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panoramacited by 67
  18. 2020DeepSphere: a graph-based spherical CNNcited by 65
  19. 2019OpenVSLAM: A Versatile Visual SLAM Frameworkcited by 64
  20. 2019Corners for Layout: End-to-End Layout Recovery from 360 Imagescited by 55
  21. 2019Spherical View Synthesis for Self-Supervised 360 Depth Estimationcited by 51
  22. 2021LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Renderingcited by 42
  23. 2018Self-Supervised Learning of Depth and Camera Motion from 360deg Videoscited by 40
  24. 2018SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Imagescited by 40
  25. 2019360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volumecited by 40
  26. 2019Monocular Depth Estimation: A Surveycited by 37
  27. 2018Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360deg Panoramic Imagerycited by 34
  28. 2019Spherical U-Net on Cortical Surfaces: Methods and Applicationscited by 34
  29. 2017Im2Pano3D: Extrapolating 360 Structure and Semantics Beyond the Field of Viewcited by 25
  30. 2019Pano Popups: Indoor 3D Reconstruction with a Plane-Aware Networkcited by 25
  31. 2018A System for Acquiring, Processing, and Rendering Panoramic Light Field Stills for Virtual Realitycited by 24
  32. 2019SweepNet: Wide-baseline Omnidirectional Depth Estimationcited by 24
  33. 2018Omnidirectional CNN for Visual Place Recognition and Navigationcited by 22
  34. 2019OmniMVS: End-to-End Learning for Omnidirectional Stereo Matchingcited by 21
  35. 2016Pano2CAD: Room Layout From A Single Panorama Imagecited by 20
  36. 2020OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systemscited by 18
  37. 2017Graph-Based Classification of Omnidirectional Imagescited by 16
  38. 2019Convolutions on Spherical Imagescited by 12

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 2.

11 papers.

  1. 2025A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Visioncited by 23
  2. 2025Conditional Panoramic Image Generation via Masked Autoregressive Modelingcited by 9
  3. 2025EDM: Equirectangular Projection-Oriented Dense Kernelized Feature Matchingcited by 7
  4. 2025RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimizationcited by 4
  5. 2024Elite360M: Efficient 360 Multi-task Learning via Bi-projection Fusion and Cross-task Collaborationcited by 3
  6. 2024ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformerscited by 2
  7. 2024Deep Spherical Superpixelscited by 0
  8. 2025A Survey on Text-Driven 360-Degree Panorama Generationcited by 0
  9. 2025Generalized Shortest Path-based Superpixels for 3D Spherical Image Segmentationcited by 0
  10. 2026Probing CLIP's Comprehension of 360-Degree Textual and Visual Semanticscited by 0
  11. 2026Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modelingcited by 0