mlsanityData to 7 Oct 2026

Helvipad: A Real-World Dataset for Omnidirectional Stereo Depth Estimation

arXiv 2411.18335 ↗First posted 27 Nov 2024cs.CV cs.AI cs.ROcited by 6

Not on the map. Why?

Abstract (arXiv)

Despite progress in stereo depth estimation, omnidirectional imaging remains underexplored, mainly due to the lack of appropriate data. We introduce Helvipad, a real-world dataset for omnidirectional stereo depth estimation, featuring 40K video frames from video sequences across diverse environments, including crowded indoor and outdoor scenes with various lighting conditions. Collected using two 360deg cameras in a top-bottom setup and a LiDAR sensor, the dataset includes accurate depth and disparity labels by projecting 3D point clouds onto equirectangular images. Additionally, we provide an augmented training set with an increased label density by using depth completion. We benchmark leading stereo depth estimation models for both standard and omnidirectional images. The results show that while recent stereo methods perform decently, a challenge persists in accurately estimating depth in omnidirectional imaging. To address this, we introduce necessary adaptations to stereo models, leading to improved performance.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 45 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

35 of 45 references.

References · 35

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  3. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  4. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  5. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  6. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  7. 2018Pyramid Stereo Matching Networkcited by 486
  8. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  9. 2017Joint 2D-3D-Semantic Data for Indoor Scene Understandingcited by 404
  10. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  11. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  12. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
23 more
  1. 2019Group-wise Correlation Stereo Networkcited by 235
  2. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  3. 2023Iterative Geometry Encoding Volume for Stereo Matchingcited by 188
  4. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  5. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
  6. 2015Stereo Matching by Training a Convolutional Neural Network to Compare Image Patchescited by 151
  7. 2022KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Waycited by 132
  8. 2022PanoFormer: Panorama Transformer for Indoor 360 Depth Estimationcited by 94
  9. 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
  10. 2013Indoor Semantic Segmentation using depth informationcited by 56
  11. 2019Orientation-aware Semantic Segmentation on Icosahedron Spherescited by 52
  12. 2019360SD-Net: 360deg Stereo Depth Estimation with Learnable Cost Volumecited by 40
  13. 2022Deep Learning for Omnidirectional Vision: A Survey and New Perspectivescited by 38
  14. 2024Neural Markov Random Field for Stereo Matchingcited by 38
  15. 2021Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimationcited by 37
  16. 2019DS-PASS: Detail-Sensitive Panoramic Annular Semantic Segmentation through SwaftNet for Surrounding Sensingcited by 30
  17. 2019SweepNet: Wide-baseline Omnidirectional Depth Estimationcited by 24
  18. 2024Playing to Vision Foundation Model's Strengths in Stereo Matchingcited by 23
  19. 2022Cylin-Painting: Seamless 360textdegree Panoramic Image Outpainting and Beyondcited by 12
  20. 2023Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervisioncited by 8
  21. 2024Temporally Consistent Stereo Matchingcited by 8
  22. 2022A Survey of Robotic Harvesting Systems and Enabling Technologiescited by 6
  23. 2023An Efficient Probabilistic Solution to Mapping Errors in LiDAR-Camera Fusion for Autonomous Vehiclescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2026Holo360D: A Large-Scale Real-World Dataset with Continuous Trajectories for Advancing Panoramic 3D Reconstruction and Beyondcited by 7
  2. 2025Boosting Omnidirectional Stereo Matching with a Pre-trained Depth Foundation Modelcited by 1
  3. 2026WideDepth: Millimeter-Accurate Benchmark for Fisheye Depth Estimationcited by 1
  4. 2026H-OmniStereo: Zero-Shot Omnidirectional Stereo Matching with Heading-Aligned Normal Priorscited by 0
  5. 2026OmniDS: Dual-Stream Context Fusion for Omnidirectional Depth from Fisheye Camerascited by 0
  6. 2026Revisiting Matching Response and Swept Feature Volumes for Wide-baseline Omnidirectional Stereocited by 0