# Using Synthetic Data to Train Neural Networks is Model-Based Reasoning

arXiv: https://arxiv.org/abs/1703.00868 · HTML: https://mlsanity.com/paper/1703.00868/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2017.

## Built on by (most cited first, 8 of 8)

- [Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models](https://mlsanity.com/paper/2504.17660/) (2025)
- [Can LLMs Replace Economic Choice Prediction Labs? The Case of Language-based Persuasion Games](https://mlsanity.com/paper/2401.17435/) (2024)
- [Graphically Structured Diffusion Models](https://mlsanity.com/paper/2210.11633/) (2022)
- [Convolution, aggregation and attention based deep neural networks for accelerating simulations in mechanics](https://mlsanity.com/paper/2212.01386/) (2022)
- [Counteracting Dark Web Text-Based CAPTCHA with Generative Adversarial Learning for Proactive Cyber Threat Intelligence](https://mlsanity.com/paper/2201.02799/) (2022)
- [Pallet Detection from Synthetic Data Using Game Engines](https://mlsanity.com/paper/2304.03602/) (2023)
- [Towards Daily High-resolution Inundation Observations using Deep Learning and EO](https://mlsanity.com/paper/2208.09135/) (2022)
- [Digital Twin Framework for Optimal and Autonomous Decision-Making in Cyber-Physical Systems: Enhancing Reliability and Adaptability in the Oil and Gas Industry](https://mlsanity.com/paper/2311.12755/) (2023)

## Abstract (arXiv)

We draw a formal connection between using synthetic training data to optimize neural network parameters and approximate, Bayesian, model-based reasoning. In particular, training a neural network using synthetic data can be viewed as learning a proposal distribution generator for approximate inference in the synthetic-data generative model. We demonstrate this connection in a recognition task where we develop a novel Captcha-breaking architecture and train it using synthetic data, demonstrating both state-of-the-art performance and a way of computing task-specific posterior uncertainty. Using a neural network trained this way, we also demonstrate successful breaking of real-world Captchas currently used by Facebook and Wikipedia. Reasoning from these empirical results and drawing connections with Bayesian modeling, we discuss the robustness of synthetic data results and suggest important considerations for ensuring good neural network generalization when training with synthetic data.