# What Makes Good Synthetic Training Data for Zero-Shot Stereo Matching?

arXiv: https://arxiv.org/abs/2504.16930 · HTML: https://mlsanity.com/paper/2504.16930/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 23 Apr 2025.

## Builds on (most cited first, 30 of 31)

- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [DynamicStereo: Consistent Dynamic Depth from Stereo Videos](https://mlsanity.com/paper/2305.02296/) (2023)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](https://mlsanity.com/paper/1804.06534/) (2018)
- [Occlusions, Motion and Depth Boundaries with a Generic Network for Disparity, Optical Flow or Scene Flow Estimation](https://mlsanity.com/paper/1808.01838/) (2018)
- [IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching](https://mlsanity.com/paper/2409.00638/) (2024)
- [Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail](https://mlsanity.com/paper/2412.04472/) (2024)
- [What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?](https://mlsanity.com/paper/1801.06397/) (2018)
- [Neural Markov Random Field for Stereo Matching](https://mlsanity.com/paper/2403.11193/) (2024)
- [Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces](https://mlsanity.com/paper/2301.08245/) (2023)
- [Stereo Matching in Time: 100+ FPS Video Stereo Matching for Extended Reality](https://mlsanity.com/paper/2309.04183/) (2023)
- [Match Stereo Videos via Bidirectional Alignment](https://mlsanity.com/paper/2409.20283/) (2024)
- [Self-supervised AutoFlow](https://mlsanity.com/paper/2212.01762/) (2022)

## Built on by (most cited first, 7 of 7)

- [WAFT-Stereo: Warping-Alone Field Transforms for Stereo Matching](https://mlsanity.com/paper/2603.24836/) (2026)
- [MatchAttention: Embedding Explicit Matching Constraints into Attention for Efficient Stereo Matching](https://mlsanity.com/paper/2510.14260/) (2025)
- [ProcFunc: Function-Oriented Abstractions for Procedural 3D Generation in Python](https://mlsanity.com/paper/2604.26943/) (2026)
- [Independent Samples, Correlated Variance A Learnable Cross-View Cue in Path-Traced Stereo Data](https://mlsanity.com/paper/2606.25483/) (2026)
- [NBS: No Bias Stereo](https://mlsanity.com/paper/2608.28933/) (2026)
- [SatUnreal: A High-Precision Synthetic Dataset for Satellite Stereo Matching via Unreal Engine](https://mlsanity.com/paper/2609.27442/) (2026)
- [FAST: Flow Any Scene Transformer](https://mlsanity.com/paper/2609.39748/) (2026)

## Abstract (arXiv)

Synthetic datasets are a crucial ingredient for training stereo matching networks, but the question of what makes a stereo dataset effective remains underexplored. We investigate the design space of synthetic datasets by varying the parameters of a procedural dataset generator, and report the effects on zero-shot stereo matching performance using standard benchmarks. We validate our findings by collecting the best settings and creating a large-scale dataset. Training only on this dataset achieves better performance than training on a mixture of widely used datasets, and is competitive with training on the FoundationStereo dataset, with the additional benefit of open-source generation code and an accompanying parameter analysis to enable further research. We open-source our system at https://github.com/princeton-vl/InfinigenStereo to enable further research on procedural stereo datasets.