mlsanityData to 7 Oct 2026

Robust and Flexible Omnidirectional Depth Estimation with Multiple 360-degree Cameras

arXiv 2409.14766 ↗First posted 23 Sep 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Omnidirectional depth estimation has received much attention from researchers in recent years. However, challenges arise due to camera soiling and variations in camera layouts, affecting the robustness and flexibility of the algorithm. In this paper, we use the geometric constraints and redundant information of multiple 360-degree cameras to achieve robust and flexible multi-view omnidirectional depth estimation. We implement two algorithms, in which the two-stage algorithm obtains initial depth maps by pairwise stereo matching of multiple cameras and fuses the multiple depth maps to achieve the final depth estimation; the one-stage algorithm adopts spherical sweeping based on hypothetical depths to construct a uniform spherical matching cost of the multi-camera images and obtain the depth. Additionally, a generalized epipolar equirectangular projection is introduced to simplify the spherical epipolar constraints. To overcome panorama distortion, a spherical feature extractor is implemented. Furthermore, a synthetic 360-degree dataset consisting of 12K road scene panoramas and 3K ground truth depth maps is presented to train and evaluate 360-degree depth estimation algorithms. Our dataset takes soiled camera lenses and glare into consideration, which is more consistent with the real-world environment. Experiments show that our two algorithms achieve state-of-the-art performance, accurately predicting depth maps even when provided with soiled panorama inputs. The flexibility of the algorithms is experimentally validated in terms of camera layouts and numbers.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 68 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

42 of 68 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  6. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  7. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  8. 2018Pyramid Stereo Matching Networkcited by 486
  9. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  10. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  11. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  12. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
30 more
  1. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  2. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 252
  3. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 242
  4. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  5. 2022Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlationcited by 197
  6. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  7. 2021CFNet: Cascade and Fused Cost Volume for Robust Stereo Matchingcited by 153
  8. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151
  9. 2019Point-Based Multi-View Stereo Networkcited by 147
  10. 2022Attention Concatenation Volume for Accurate and Efficient Stereo Matchingcited by 130
  11. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 98
  12. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  13. 2020Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checkingcited by 81
  14. 2017Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matchingcited by 63
  15. 2022OmniFusion: 360 Monocular Depth Estimation via Geometry-Aware Fusioncited by 60
  16. 2019Spherical View Synthesis for Self-Supervised 360 Depth Estimationcited by 51
  17. 2022Multiview Stereo with Cascaded Epipolar RAFTcited by 51
  18. 2019Pyramid Multi-view Stereo Net with Self-adaptive View Aggregationcited by 44
  19. 2018Self-Supervised Learning of Depth and Camera Motion from 360deg Videoscited by 40
  20. 2019360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volumecited by 40
  21. 2019EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detectioncited by 39
  22. 2022BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimationcited by 35
  23. 2018Eliminating the Blind Spot: Adapting 3D Object Detection and Monocular Depth Estimation to 360deg Panoramic Imagerycited by 34
  24. 2019SweepNet: Wide-baseline Omnidirectional Depth Estimationcited by 24
  25. 2021On the confidence of stereo matching in a deep-learning era: a quantitative evaluationcited by 23
  26. 2019OmniMVS: End-to-End Learning for Omnidirectional Stereo Matchingcited by 21
  27. 2020ODE-CNN: Omnidirectional Depth Extension Networkscited by 18
  28. 2022360 Depth Estimation in the Wild -- The Depth360 Dataset and the SegFuse Networkcited by 15
  29. 2023DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volumecited by 8
  30. 2021AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approachcited by 7

Built on by

Not cited yet.