# Does Training on Synthetic Data Make Models Less Robust?

arXiv: https://arxiv.org/abs/2502.07164 · HTML: https://mlsanity.com/paper/2502.07164/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 15 of 15)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://mlsanity.com/paper/1902.01007/) (2019)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [Self-Consuming Generative Models Go MAD](https://mlsanity.com/paper/2307.01850/) (2023)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models](https://mlsanity.com/paper/2102.04130/) (2021)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation](https://mlsanity.com/paper/2104.08678/) (2021)
- [How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse](https://mlsanity.com/paper/2404.05090/) (2024)
- [This dataset does not exist: training models from generated images](https://mlsanity.com/paper/1911.02888/) (2019)
- [Can segmentation models be trained with fully synthetically generated data?](https://mlsanity.com/paper/2209.08256/) (2022)
- [Retrieval-guided Counterfactual Generation for QA](https://mlsanity.com/paper/2110.07596/) (2021)
- [Real Risks of Fake Data: Synthetic Data, Diversity-Washing and Consent Circumvention](https://mlsanity.com/paper/2405.01820/) (2024)

## Built on by (most cited first, 3 of 3)

- [Modelling the Morphology of Verbal Paradigms: A Case Study in the Tokenization of Turkish and Hebrew](https://mlsanity.com/paper/2602.05648/) (2026)
- [Datasets for Verb Alternations across Languages: BLM Templates and Data Augmentation Strategies](https://mlsanity.com/paper/2603.15295/) (2026)
- [Selecting Language Models for Social Science: Start Small, Start Open, and Validate](https://mlsanity.com/paper/2601.10926/) (2026)

## Abstract (arXiv)

An increasingly common practice is to train large language models (LLMs) using synthetic data. Often this synthetic data is produced by the same or similar LLMs as those it is being used to train. This raises the question of whether the synthetic data might in fact exacerbate certain "blindspots" by reinforcing heuristics that the LLM already encodes. In this paper, we conduct simulated experiments on the natural language inference (NLI) task with Llama-2-7B-hf models. We use MultiNLI as the general task and HANS, a targeted evaluation set designed to measure the presence of specific heuristic strategies for NLI, as our "blindspot" task. Our goal is to determine whether performance disparities between the general and blind spot tasks emerge. Our results indicate that synthetic data does not reinforce blindspots in the way we expected. Specifically, we see that, while fine-tuning with synthetic data doesn't necessarily reduce the use of the heuristic, it also does not make it worse as we hypothesized.