mlsanityData to 14 Oct 2026

Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Model

arXiv 2503.23502 ↗First posted 30 Mar 2025cs.CV cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Omnidirectional depth perception is essential for mobile robotics applications that require scene understanding across a full 360deg field of view. Camera-based setups offer a cost-effective option by using stereo depth estimation to generate dense, high-resolution depth maps without relying on expensive active sensing. However, existing omnidirectional stereo matching approaches achieve only limited depth accuracy across diverse environments, depth ranges, and lighting conditions, due to the scarcity of real-world data. We present DFI-OmniStereo, a novel omnidirectional stereo matching method that leverages a large-scale pre-trained foundation model for relative monocular depth estimation within an iterative optimization-based stereo matching architecture. We introduce a dedicated two-stage training strategy to utilize the relative monocular depth features for our omnidirectional stereo matching before scale-invariant fine-tuning. DFI-OmniStereo achieves state-of-the-art results on the real-world Helvipad dataset, reducing disparity MAE by approximately 16% compared to the previous best omnidirectional stereo method.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 48 of its 67 references that are in the map. First posted on arXiv 30 Mar 2025.

Builds on

48 of 67 references.

References · 48

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2021Vision Transformers for Dense Predictioncited by 1,803
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  7. 2024Depth Anything V2cited by 1,422
  8. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  9. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  10. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  11. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  12. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
36 more
  1. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  2. 2018Pyramid Stereo Matching Networkcited by 489
  3. 2018Spherical CNNscited by 456
  4. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 406
  5. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  6. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 379
  7. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 360
  8. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 353
  9. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  10. 2019GA-Net: Guided Aggregation Net for End-to-end Stereo Matchingcited by 244
  11. 2019Group-wise Correlation Stereo Networkcited by 235
  12. 2022Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlationcited by 197
  13. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 190
  14. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 174
  15. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 173
  16. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166
  17. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151
  18. 2014Deep Convolutional Neural Fields for Depth Estimation from a Single Imagecited by 106
  19. 2024Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matchingcited by 99
  20. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 95
  21. 2019Tangent Images for Mitigating Spherical Distortioncited by 79
  22. 2018Cube Padding for Weakly-Supervised Saliency Prediction in 360deg Videoscited by 66
  23. 2024MoCha-Stereo: Motif Channel Attention Network for Stereo Matchingcited by 48
  24. 2023EGformer: Equirectangular Geometry-biased Transformer for 360 Depth Estimationcited by 43
  25. 2019360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volumecited by 40
  26. 2019SweepNet: Wide-baseline Omnidirectional Depth Estimationcited by 24
  27. 2024Playing to Vision Foundation Model's Strengths in Stereo Matchingcited by 23
  28. 2024A Survey on Deep Stereo Matching in the Twentiescited by 23
  29. 2018Omnidirectional CNN for Visual Place Recognition and Navigationcited by 22
  30. 2019OmniMVS: End-to-End Learning for Omnidirectional Stereo Matchingcited by 21
  31. 2023HEAL-SWIN: A Vision Transformer On The Spherecited by 20
  32. 2020360^circ Depth Estimation from Multiple Fisheye Images with Origami Crown Representation of Icosahedroncited by 10
  33. 2023Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervisioncited by 8
  34. 2024RomniStereo: Recurrent Omnidirectional Stereo Matchingcited by 8
  35. 2024Dusk Till Dawn: Self-supervised Nighttime Stereo Depth Estimation using Visual Foundation Modelscited by 6
  36. 2024Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priorscited by 0