# How much real data do we actually need: Analyzing object detection performance using synthetic and real data

arXiv: https://arxiv.org/abs/1907.07061 · HTML: https://mlsanity.com/paper/1907.07061/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2019.

## Built on by (most cited first, 29 of 29)

- [Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2304.10410/) (2023)
- [Efficacy of Synthetic Data as a Benchmark](https://mlsanity.com/paper/2409.11968/) (2024)
- [Sim2Air - Synthetic aerial dataset for UAV monitoring](https://mlsanity.com/paper/2110.05145/) (2021)
- [VersatileGait: A Large-Scale Synthetic Gait Dataset with Fine-GrainedAttributes and Complicated Scenarios](https://mlsanity.com/paper/2101.01394/) (2021)
- [Advancing Fine-Grained Classification by Structure and Subject Preserving Augmentation](https://mlsanity.com/paper/2406.14551/) (2024)
- [Analysis of Training Object Detection Models with Synthetic Data](https://mlsanity.com/paper/2211.16066/) (2022)
- [StorSeismic: A new paradigm in deep learning for seismic processing](https://mlsanity.com/paper/2205.00222/) (2022)
- [Quantifying the LiDAR Sim-to-Real Domain Shift: A Detailed Investigation Using Object Detectors and Analyzing Point Clouds at Target-Level](https://mlsanity.com/paper/2303.01899/) (2023)
- [Object Detector Differences when using Synthetic and Real Training Data](https://mlsanity.com/paper/2312.00694/) (2023)
- [Reducing the Amount of Real World Data for Object Detector Training with Synthetic Data](https://mlsanity.com/paper/2202.00632/) (2022)
- [Synthetic Data Generation for Bridging Sim2Real Gap in a Production Environment](https://mlsanity.com/paper/2311.11039/) (2023)
- [Autoencoder Based Inter-Vehicle Generalization for In-Cabin Occupant Classification](https://mlsanity.com/paper/2105.03164/) (2021)
- [Synthetic Industrial Object Detection: GenAI vs. Feature-Based Methods](https://mlsanity.com/paper/2511.23241/) (2025)
- [TRoVE: Transforming Road Scene Datasets into Photorealistic Virtual Environments](https://mlsanity.com/paper/2208.07943/) (2022)
- [You Only Crash Once: Improved Object Detection for Real-Time, Sim-to-Real Hazardous Terrain Detection and Classification for Autonomous Planetary Landings](https://mlsanity.com/paper/2303.04891/) (2023)
- [Data and Knowledge for Overtaking Scenarios in Autonomous Driving](https://mlsanity.com/paper/2305.19421/) (2023)
- [Deep Aramaic: Towards a Synthetic Data Paradigm Enabling Machine Learning in Epigraphy](https://mlsanity.com/paper/2310.07310/) (2023)
- [Bayesian Detector Combination for Object Detection with Crowdsourced Annotations](https://mlsanity.com/paper/2407.07958/) (2024)
- [Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventions](https://mlsanity.com/paper/2409.15927/) (2024)
- [Development of Hybrid Artificial Intelligence Training on Real and Synthetic Data: Benchmark on Two Mixed Training Strategies](https://mlsanity.com/paper/2506.24093/) (2025)
- [Realistically distributing object placements in synthetic training data improves the performance of vision-based object detection models](https://mlsanity.com/paper/2305.14621/) (2023)
- [Is AI currently capable of identifying wild oysters? A comparison of human annotators against the AI model, ODYSSEE](https://mlsanity.com/paper/2505.03108/) (2025)
- [Synthetic Dataset Generation for Autonomous Mobile Robots Using 3D Gaussian Splatting for Vision Training](https://mlsanity.com/paper/2506.05092/) (2025)
- [Can We Challenge Open-Vocabulary Object Detectors with Generated Content in Street Scenes?](https://mlsanity.com/paper/2506.23751/) (2025)
- [Does Synthetic Layered Design Data Benefit Layered Design Decomposition?](https://mlsanity.com/paper/2605.15167/) (2026)
- [How many radiographs are needed to re-train a deep learning system for object detection?](https://mlsanity.com/paper/2210.08734/) (2022)
- [OSSA: Unsupervised One-Shot Style Adaptation](https://mlsanity.com/paper/2410.00900/) (2024)
- [UrbanTwin: Building High-Fidelity Digital Twins for Sim2Real LiDAR Perception and Evaluation](https://mlsanity.com/paper/2509.02903/) (2025)
- [High-Fidelity Digital Twins for Bridging the Sim2Real Gap in LiDAR-Based ITS Perception](https://mlsanity.com/paper/2509.02904/) (2025)

## Abstract (arXiv)

In recent years, deep learning models have resulted in a huge amount of progress in various areas, including computer vision. By nature, the supervised training of deep models requires a large amount of data to be available. This ideal case is usually not tractable as the data annotation is a tremendously exhausting and costly task to perform. An alternative is to use synthetic data. In this paper, we take a comprehensive look into the effects of replacing real data with synthetic data. We further analyze the effects of having a limited amount of real data. We use multiple synthetic and real datasets along with a simulation tool to create large amounts of cheaply annotated synthetic data. We analyze the domain similarity of each of these datasets. We provide insights about designing a methodological procedure for training deep networks using these datasets.