mlsanityData to 7 Oct 2026

MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas

arXiv 2408.01653 ↗First posted 3 Aug 2024cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Omnidirectional depth estimation presents a significant challenge due to the inherent distortions in panoramic images. Despite notable advancements, the impact of projection methods remains underexplored. We introduce Multi-Cylindrical Panoramic Depth Estimation (MCPDepth), a novel two-stage framework designed to enhance omnidirectional depth estimation through stereo matching across multiple cylindrical panoramas. MCPDepth initially performs stereo matching using cylindrical panoramas, followed by a robust fusion of the resulting depth maps from different views. Unlike existing methods that rely on customized kernels to address distortions, MCPDepth utilizes standard network components, facilitating seamless deployment on embedded devices while delivering exceptional performance. To effectively address vertical distortions in cylindrical panoramas, MCPDepth incorporates a circular attention module, significantly expanding the receptive field beyond traditional convolutions. We provide a comprehensive theoretical and experimental analysis of common panoramic projections-spherical, cylindrical, and cubic-demonstrating the superior efficacy of cylindrical projection. Our method improves the mean absolute error (MAE) by 18.8% on the outdoor dataset Deep360 and by 19.9% on the real dataset 3D60. This work offers practical insights for other tasks and real-world applications, establishing a new paradigm in omnidirectional depth estimation. The code is available at https://github.com/Qjizhi/MCPDepth.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 67 references that are in the map. First posted on arXiv 3 Aug 2024.

Builds on

51 of 67 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  8. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  9. 2018Self-Attention Generative Adversarial Networkscited by 809
  10. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  11. 2018Image Transformercited by 770
  12. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
38 more
  1. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  2. 2018Pyramid Stereo Matching Networkcited by 486
  3. 2019Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matchingcited by 414
  4. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  5. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  6. 2021An End-to-End Transformer Model for 3D Object Detectioncited by 329
  7. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  8. 2019Group-wise Correlation Stereo Networkcited by 235
  9. 2019Local Relation Networks for Image Recognitioncited by 210
  10. 2021Rethinking and Improving Relative Position Encoding for Vision Transformercited by 206
  11. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  12. 2022Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlationcited by 197
  13. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  14. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 188
  15. 2019Cost Volume Pyramid Based Depth Inference for Multi-View Stereocited by 154
  16. 2019Point-Based Multi-View Stereo Networkcited by 147
  17. 2020HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matchingcited by 120
  18. 2018OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramascited by 98
  19. 2024Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matchingcited by 98
  20. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  21. 2021UniFuse: Unidirectional Fusion for 360circ Panorama Depth Estimationcited by 91
  22. 2019Spherical CNNs on Unstructured Gridscited by 76
  23. 2014Computing the Stereo Matching Cost with a Convolutional Neural Networkcited by 74
  24. 2017Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matchingcited by 63
  25. 2021IterMVS: Iterative Probability Estimation for Efficient Multi-View Stereocited by 56
  26. 2019Corners for Layout: End-to-End Layout Recovery from 360 Imagescited by 55
  27. 2019Spherical View Synthesis for Self-Supervised 360 Depth Estimationcited by 51
  28. 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimationcited by 42
  29. 2019360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volumecited by 40
  30. 2022STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenescited by 35
  31. 2019SweepNet: Wide-baseline Omnidirectional Depth Estimationcited by 24
  32. 2021On the confidence of stereo matching in a deep-learning era: a quantitative evaluationcited by 23
  33. 2019OmniMVS: End-to-End Learning for Omnidirectional Stereo Matchingcited by 21
  34. 2023Global Occlusion-Aware Transformer for Robust Stereo Matchingcited by 19
  35. 2020ODE-CNN: Omnidirectional Depth Extension Networkscited by 18
  36. 2023PanoSwin: a Pano-style Swin Transformer for Panorama Understandingcited by 18
  37. 20203D Object Detection from a Single Fisheye Image Without a Single Fisheye Training Imagecited by 13
  38. 2024SGFormer: Spherical Geometry Transformer for 360 Depth Estimationcited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.