mlsanityData to 14 Oct 2026

Deep Learning for Omnidirectional Vision: A Survey and New Perspectives

arXiv 2205.10468 ↗First posted 21 May 2022cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

Omnidirectional image (ODI) data is captured with a 360x180 field-of-view, which is much wider than the pinhole cameras and contains richer spatial information than the conventional planar images. Accordingly, omnidirectional vision has attracted booming attention due to its more advantageous performance in numerous applications, such as autonomous driving and virtual reality. In recent years, the availability of customer-level 360 cameras has made omnidirectional vision more popular, and the advance of deep learning (DL) has significantly sparked its research and applications. This paper presents a systematic and comprehensive review and analysis of the recent progress in DL methods for omnidirectional vision. Our work covers four main contents: (i) An introduction to the principle of omnidirectional imaging, the convolution methods on the ODI, and datasets to highlight the differences and difficulties compared with the 2D planar image data; (ii) A structural and hierarchical taxonomy of the DL methods for omnidirectional vision; (iii) A summarization of the latest novel learning strategies and applications; (iv) An insightful discussion of the challenges and open problems by highlighting the potential research directions to trigger more research in the community.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 124 of its 243 references that are in the map. First posted on arXiv 21 May 2022.

Builds on

124 of 243 references.

References · 124

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022Generative Adversarial Networkscited by 5,627
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  5. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  6. 2021Vision Transformers for Dense Predictioncited by 1,803
  7. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  9. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  10. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  11. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  12. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
38 more
  1. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  2. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 472
  3. 2018Spherical CNNscited by 456
  4. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  5. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  6. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 286
  7. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 267
  8. 2017Learning SO(3) Equivariant Representations with Spherical CNNscited by 248
  9. 2020VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrievalcited by 189
  10. 2016What do different evaluation metrics tell us about saliency models?cited by 185
  11. 2019Gaze360: Physically Unconstrained Gaze Estimation in the Wildcited by 174
  12. 2022TransGeo: Transformer Is All You Need for Cross-view Image Geo-localizationcited by 168
  13. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 161
  14. 2020Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matchingcited by 126
  15. 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
  16. 2020DeepFactors: Real-Time Probabilistic Dense Monocular SLAMcited by 117
  17. 2021Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localizationcited by 113
  18. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 108
  19. 2020HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Featurescited by 105
  20. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  21. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 99
  22. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 95
  23. 2019Bridging the Domain Gap for Ground-to-Aerial Image Matchingcited by 95
  24. 2020Learning Representations from Audio-Visual Spatial Alignmentcited by 95
  25. 2018Self-Supervised Generation of Spatial Audio for 360 Videocited by 93
  26. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 92
  27. 2016Predicting Ground-Level Scene Layout from Aerial Imagerycited by 90
  28. 2021Pano-AVQA: Grounded Audio-Visual Question Answering on 360^circ Videoscited by 86
  29. 2019Deep Parametric Indoor Lighting Estimationcited by 83
  30. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  31. 20213D Semantic Scene Completion: a Surveycited by 74
  32. 2022Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentationcited by 72
  33. 2018DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panoramacited by 68
  34. 2018Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videoscited by 66
  35. 2020DeepSphere: a graph-based spherical CNNcited by 66
  36. 2017SalGAN: Visual Saliency Prediction with Generative Adversarial Networkscited by 65
  37. 2022OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusioncited by 61
  38. 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60

50 most cited of 124 All 124 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 11; 2024: 15; 2025: 6; 2026: 5.

38 papers.

Sort:

By area:

  1. 2024360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Modelcited by 58
  2. 2023Both Style and Distortion Matter: Dual-Path Unsupervised Domain Adaptation for Panoramic Semantic Segmentationcited by 47
  3. 2022Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentationcited by 46
  4. 2023Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentationcited by 38
  5. 2023HRDFuse: Monocular 360degDepth Estimation by Collaboratively Learning Holistic-with-Regional Depth Distributionscited by 37
  6. 2024A Survey on Generative AI and LLM for Video Generation, Understanding, and Streamingcited by 37
  7. 2024Semantics, Distortion, and Style Matter: Towards Source-free UDA for Panoramic Segmentationcited by 34
  8. 2024Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusioncited by 31
  9. 2024GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentationcited by 28
  10. 2024360SFUDA++: Towards Source-free UDA for Panoramic Segmentation by Learning Reliable Category Prototypescited by 28
  11. 2025OmniSAM: Omnidirectional Segment Anything Model for UDA in Panoramic Semantic Segmentationcited by 28
  12. 2023OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformercited by 26
26 more
  1. 2023SGAT4PASS: Spherical Geometry-Aware Transformer for PAnoramic Semantic Segmentationcited by 26
  2. 2023Single Frame Semantic Segmentation Using Multi-Modal Spherical Imagescited by 18
  3. 2023PanoVOS: Bridging Non-panoramic and Panoramic Views with Transformer for Video Segmentationcited by 18
  4. 2024Open Panoramic Segmentationcited by 17
  5. 2023Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformerscited by 13
  6. 2024GoodSAM++: Bridging Domain and Capacity Gaps via Segment Anything Model for Panoramic Semantic Segmentationcited by 11
  7. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  8. 2023OPDN: Omnidirectional Position-aware Deformable Network for Omnidirectional Image Super-Resolutioncited by 9
  9. 2024Occlusion-Aware Seamless Segmentationcited by 7
  10. 2026PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learningcited by 7
  11. 2024Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimationcited by 6
  12. 2025Seam360GS: Seamless 360deg Gaussian Splatting from Real-World Omnidirectional Imagescited by 6
  13. 2023Applications of Deep Learning for Top-View Omnidirectional Imaging: A Surveycited by 3
  14. 2024Elite360M: Efficient 360 Multi-task Learning via Bi-projection Fusion and Cross-task Collaborationcited by 3
  15. 2024RealOSR: Latent Guidance Boosts Diffusion-based Real-world Omnidirectional Image Super-Resolutionscited by 3
  16. 2024Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolutioncited by 2
  17. 2026SO3UFormer: Learning Intrinsic Spherical Features for Rotation-Robust Panoramic Dense Predictioncited by 2
  18. 2024Learning High-Quality Navigation and Zooming on Omnidirectional Images in Virtual Realitycited by 1
  19. 2024CRF360D: Monocular 360 Depth Estimation via Spherical Fully-Connected CRFscited by 1
  20. 2025Omnidirectional Video Super-Resolution using Deep Learningcited by 1
  21. 2025Systematic Comparison of Projection Methods for Monocular 3D Human Pose Estimation on Fisheye Imagescited by 1
  22. 2026AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentationcited by 1
  23. 2023360^circ High-Resolution Depth Estimation via Uncertainty-aware Structural Knowledge Transfercited by 0
  24. 2025Large-FOV RGBD Imaging via Structured PSF Coding in a Bio-inspired Monocentric Systemcited by 0
  25. 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Imagecited by 0
  26. 2026SphMind: Towards Robust, Training-Free VLM-based Spatial Reasoning with a 360 Cameracited by 0