# What Makes Good Synthetic Training Data for Learning Disparity and Optical Flow Estimation?

arXiv: https://arxiv.org/abs/1801.06397 · HTML: https://mlsanity.com/paper/1801.06397/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jan 2018.

## Built on by (most cited first, 30 of 44)

- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [Learning to See by Looking at Noise](https://mlsanity.com/paper/2106.05963/) (2021)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration](https://mlsanity.com/paper/2102.12182/) (2021)
- [LRM-Zero: Training Large Reconstruction Models with Synthesized Data](https://mlsanity.com/paper/2406.09371/) (2024)
- [Imbalanced Graph Classification via Graph-of-Graph Neural Networks](https://mlsanity.com/paper/2112.00238/) (2021)
- [On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks](https://mlsanity.com/paper/2303.14840/) (2023)
- [Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image](https://mlsanity.com/paper/2110.05655/) (2021)
- [RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos](https://mlsanity.com/paper/2207.11075/) (2022)
- [Is my Depth Ground-Truth Good Enough? HAMMER -- Highly Accurate Multi-Modal Dataset for DEnse 3D Scene Regression](https://mlsanity.com/paper/2205.04565/) (2022)
- [SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo](https://mlsanity.com/paper/2106.16118/) (2021)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenes](https://mlsanity.com/paper/2404.06050/) (2024)
- [H-Net: Unsupervised Attention-based Stereo Depth Estimation Leveraging Epipolar Geometry](https://mlsanity.com/paper/2104.11288/) (2021)
- [CroMo: Cross-Modal Learning for Monocular Depth Estimation](https://mlsanity.com/paper/2203.12485/) (2022)
- [MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation](https://mlsanity.com/paper/2211.06018/) (2022)
- [Self-supervised AutoFlow](https://mlsanity.com/paper/2212.01762/) (2022)
- [WHU-Stereo: A Challenging Benchmark for Stereo Matching of High-Resolution Satellite Images](https://mlsanity.com/paper/2206.02342/) (2022)
- [Semi-Supervised Learning of Optical Flow by Flow Supervisor](https://mlsanity.com/paper/2207.10314/) (2022)
- [CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameters](https://mlsanity.com/paper/2110.14347/) (2021)
- [What Makes Good Synthetic Training Data for Zero-Shot Stereo Matching?](https://mlsanity.com/paper/2504.16930/) (2025)
- [Semi-synthesis: A fast way to produce effective datasets for stereo matching](https://mlsanity.com/paper/2101.10811/) (2021)
- [MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimation](https://mlsanity.com/paper/2310.04551/) (2023)
- [BlinkTrack: Feature Tracking over 80 FPS via Events and Images](https://mlsanity.com/paper/2409.17981/) (2024)
- [Neural Networks for Semantic Gaze Analysis in XR Settings](https://mlsanity.com/paper/2103.10451/) (2021)
- [H-GAN: the power of GANs in your Hands](https://mlsanity.com/paper/2103.15017/) (2021)
- [DynaDog+T: A Parametric Animal Model for Synthetic Canine Image Generation](https://mlsanity.com/paper/2107.07330/) (2021)
- [Multi-view Monocular Depth and Uncertainty Prediction with Deep SfM in Dynamic Environments](https://mlsanity.com/paper/2201.08633/) (2022)
- [TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments](https://mlsanity.com/paper/2208.07943/) (2022)

## Abstract (arXiv)

The finding that very large networks can be trained efficiently and reliably has led to a paradigm shift in computer vision from engineered solutions to learning formulations. As a result, the research challenge shifts from devising algorithms to creating suitable and abundant training data for supervised learning. How to efficiently create such training data? The dominant data acquisition method in visual recognition is based on web data and manual annotation. Yet, for many computer vision problems, such as stereo or optical flow estimation, this approach is not feasible because humans cannot manually enter a pixel-accurate flow field. In this paper, we promote the use of synthetically generated data for the purpose of training deep networks on such tasks.We suggest multiple ways to generate such data and evaluate the influence of dataset properties on the performance and generalization properties of the resulting networks. We also demonstrate the benefit of learning schedules that use different types of data at selected stages of the training process.