# Open Challenges in Deep Stereo: the Booster Dataset

arXiv: https://arxiv.org/abs/2206.04671 · HTML: https://mlsanity.com/paper/2206.04671/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 58 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 27 of 27)

- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [DSEC: A Stereo Event Camera Dataset for Driving Scenarios](https://mlsanity.com/paper/2103.06011/) (2021)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching](https://mlsanity.com/paper/2104.04314/) (2021)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [Domain-invariant Stereo Matching Networks](https://mlsanity.com/paper/1911.13287/) (2019)
- [SegStereo: Exploiting Semantic Information for Disparity Estimation](https://mlsanity.com/paper/1807.11699/) (2018)
- [Real-time self-adaptive deep stereo](https://mlsanity.com/paper/1810.05424/) (2018)
- [Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching](https://mlsanity.com/paper/1708.09204/) (2017)
- [Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation](https://mlsanity.com/paper/2008.07130/) (2020)
- [EdgeStereo: A Context Integrated Residual Pyramid Network for Stereo Matching](https://mlsanity.com/paper/1803.05196/) (2018)
- [Matching-space Stereo Networks for Cross-domain Generalization](https://mlsanity.com/paper/2010.07347/) (2020)
- [Unsupervised Domain Adaptation for Depth Prediction from Images](https://mlsanity.com/paper/1909.03943/) (2019)
- [Bridging Stereo Matching and Optical Flow via Spatiotemporal Correspondence](https://mlsanity.com/paper/1905.09265/) (2019)
- [Real-Time Semantic Stereo Matching](https://mlsanity.com/paper/1910.00541/) (2019)
- [Continual Adaptation for Deep Stereo](https://mlsanity.com/paper/2007.05233/) (2020)
- [Neural Disparity Refinement for Arbitrary Resolution Stereo](https://mlsanity.com/paper/2110.15367/) (2021)
- [Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling](https://mlsanity.com/paper/1612.04770/) (2016)

## Built on by (most cited first, 30 of 48)

- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [DEFOM-Stereo: Depth Foundation Model Based Stereo Matching](https://mlsanity.com/paper/2501.09466/) (2025)
- [Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail](https://mlsanity.com/paper/2412.04472/) (2024)
- [NeRF-Supervised Deep Stereo](https://mlsanity.com/paper/2303.17603/) (2023)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces](https://mlsanity.com/paper/2301.08245/) (2023)
- [Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions](https://mlsanity.com/paper/2407.16698/) (2024)
- [Masked Depth Modeling for Spatial Perception](https://mlsanity.com/paper/2601.17895/) (2026)
- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [MonSter++: Unified Stereo Matching, Multi-view Stereo, and Real-time Stereo with Monodepth Priors](https://mlsanity.com/paper/2501.08643/) (2025)
- [Epipolar-Free 3D Gaussian Splatting for Generalizable Novel View Synthesis](https://mlsanity.com/paper/2410.22817/) (2024)
- [Robust Synthetic-to-Real Transfer for Stereo Matching](https://mlsanity.com/paper/2403.07705/) (2024)
- [BridgeDepth: Bridging Monocular and Stereo Reasoning with Latent Alignment](https://mlsanity.com/paper/2508.04611/) (2025)
- [LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow](https://mlsanity.com/paper/2409.05688/) (2024)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [Diving into the Fusion of Monocular Priors for Generalized Stereo Matching](https://mlsanity.com/paper/2505.14414/) (2025)
- [Towards Robust Monocular Depth Estimation in Non-Lambertian Surfaces](https://mlsanity.com/paper/2408.06083/) (2024)
- [StereoAdapter: Adapting Stereo Depth Estimation to Underwater Scenes](https://mlsanity.com/paper/2509.16415/) (2025)
- [SPIdepth: Strengthened Pose Information for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2404.12501/) (2024)
- [Seeing and Seeing Through the Glass: Real and Synthetic Data for Multi-Layer Depth Estimation](https://mlsanity.com/paper/2503.11633/) (2025)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimation](https://mlsanity.com/paper/2607.02515/) (2026)
- [PromptStereo: Zero-Shot Stereo Matching via Structure and Motion Prompts](https://mlsanity.com/paper/2603.01650/) (2026)
- [UniTT-Stereo: Unified Training of Transformer for Enhanced Stereo Matching](https://mlsanity.com/paper/2409.02545/) (2024)
- [3D Visual Illusion Depth Estimation](https://mlsanity.com/paper/2505.13061/) (2025)
- [Multi-Label Stereo Matching for Transparent Scene Depth Estimation](https://mlsanity.com/paper/2505.14008/) (2025)
- [NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfaces](https://mlsanity.com/paper/2506.05815/) (2025)
- [DepthFocus: Controllable Depth Estimation for See-Through Scenes](https://mlsanity.com/paper/2511.16993/) (2025)
- [PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models](https://mlsanity.com/paper/2507.08400/) (2025)

## Abstract (arXiv)

We present a novel high-resolution and challenging stereo dataset framing indoor scenes annotated with dense and accurate ground-truth disparities. Peculiar to our dataset is the presence of several specular and transparent surfaces, i.e. the main causes of failures for state-of-the-art stereo networks. Our acquisition pipeline leverages a novel deep space-time stereo framework which allows for easy and accurate labeling with sub-pixel precision. We release a total of 419 samples collected in 64 different scenes and annotated with dense ground-truth disparities. Each sample include a high-resolution pair (12 Mpx) as well as an unbalanced pair (Left: 12 Mpx, Right: 1.1 Mpx). Additionally, we provide manually annotated material segmentation masks and 15K unlabeled samples. We evaluate state-of-the-art deep networks based on our dataset, highlighting their limitations in addressing the open challenges in stereo and drawing hints for future research.