mlsanityData to 7 Oct 2026

A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision

arXiv 2502.10444 ↗First posted 11 Feb 2025cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Omnidirectional image (ODI) data is captured with a field-of-view of 360x180, which is much wider than the pinhole cameras and captures richer surrounding environment details than the conventional perspective images. In recent years, the availability of customer-level 360 cameras has made omnidirectional vision more popular, and the advance of deep learning (DL) has significantly sparked its research and applications. This paper presents a systematic and comprehensive review and analysis of the recent progress of DL for omnidirectional vision. It delineates the distinct challenges and complexities encountered in applying DL to omnidirectional images as opposed to traditional perspective imagery. Our work covers four main contents: (i) A thorough introduction to the principles of omnidirectional imaging and commonly explored projections of ODI; (ii) A methodical review of varied representation learning approaches tailored for ODI; (iii) An in-depth investigation of optimization strategies specific to omnidirectional vision; (iv) A structural and hierarchical taxonomy of the DL methods for the representative omnidirectional vision tasks, from visual enhancement (e.g., image generation and super-resolution) to 3D geometry and motion estimation (e.g., depth and optical flow estimation), alongside the discussions on emergent research directions; (v) An overview of cutting-edge applications (e.g., autonomous driving and virtual reality), coupled with a critical discussion on prevailing challenges and open questions, to trigger more research in the community.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 179 of its 343 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

179 of 343 references.

References · 179

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  9. 2023Segment Anythingcited by 8,158
  10. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
38 more
  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  3. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  4. 2014Conditional Generative Adversarial Netscited by 2,152
  5. 2021Vision Transformers for Dense Predictioncited by 1,789
  6. 2016Generative Adversarial Imitation Learningcited by 1,576
  7. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  9. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  10. 2016Group Equivariant Convolutional Networkscited by 1,154
  11. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  12. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  13. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  14. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  15. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  16. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  17. 2019Dynamic Convolution: Attention over Convolution Kernelscited by 355
  18. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  19. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  20. 2021Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentationcited by 268
  21. 2019Gauge Equivariant Convolutional Networks and the Icosahedral CNNcited by 262
  22. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  23. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  24. 2022Structure-Aware Transformer for Graph Representation Learningcited by 191
  25. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 160
  26. 2019HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentationcited by 107
  27. 2020HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Featurescited by 104
  28. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 98
  29. 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 94
  30. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  31. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  32. 2019Deep Parametric Indoor Lighting Estimationcited by 82
  33. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  34. 2023PanoGen: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigationcited by 77
  35. 2019Spherical CNNs on Unstructured Gridscited by 76
  36. 2019Fast Spatially-Varying Indoor Lighting Estimationcited by 72
  37. 2022Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentationcited by 71
  38. 2018DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panoramacited by 67

50 most cited of 179 All 179 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 13.

23 papers.

Sort:

By area:

  1. 2025Omnidirectional Multi-Object Trackingcited by 12
  2. 2025PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Eracited by 12
  3. 2025QuaDreamer: Controllable Panoramic Video Generation for Quadruped Robotscited by 7
  4. 2026PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learningcited by 7
  5. 2026Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyondcited by 7
  6. 2025PriOr-Flow: Enhancing Primitive Panoramic Optical Flow with Orthogonal Viewcited by 4
  7. 2025Unlocking Constraints: Source-Free Occlusion-Aware Seamless Segmentationcited by 3
  8. 2025OB3D: A New Dataset for Benchmarking Omnidirectional 3D Reconstruction Using Blendercited by 2
  9. 2025OmniTrack++: Omnidirectional Multi-Object Tracking by Learning Large-FoV Trajectory Feedbackcited by 2
  10. 2026Panoramic Multimodal Semantic Occupancy Prediction for Quadruped Robotscited by 2
  11. 2025Depth Anything in 360^circ: Towards Scale Invariance in the Wildcited by 1
  12. 2026O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Roboticscited by 1
11 more
  1. 2026Argus: Metric Panoramic 3D Reconstruction for Indoor Scenescited by 1
  2. 2026AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentationcited by 1
  3. 2025A Survey on Text-Driven 360-Degree Panorama Generationcited by 0
  4. 2025Panoramic Out-of-Distribution Segmentationcited by 0
  5. 2026Spherical-GOF: Geometry-Aware Panoramic Gaussian Opacity Fields for 3D Scene Reconstructioncited by 0
  6. 2026SAP: Segment Any 4K Panoramacited by 0
  7. 2026Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentationcited by 0
  8. 2026Probing CLIP's Comprehension of 360-Degree Textual and Visual Semanticscited by 0
  9. 2026Seek to Segment: Active Perception for Panoramic Referring Segmentationcited by 0
  10. 2026InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360deg Imagecited by 0
  11. 2026SphMind: Towards Robust, Training-Free VLM-based Spatial Reasoning with a 360 Cameracited by 0