# Utilizing Large Language Models to Generate Synthetic Data to Increase the Performance of BERT-Based Neural Networks

arXiv: https://arxiv.org/abs/2405.06695 · HTML: https://mlsanity.com/paper/2405.06695/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 8 May 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Medical synthetic data and agents](https://mlsanity.com/synth/medical-synthetic-data-and-agents/)

## Builds on (most cited first, 7 of 7)

- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models](https://mlsanity.com/paper/2303.10420/) (2023)
- [Generating Datasets with Pretrained Language Models](https://mlsanity.com/paper/2104.07540/) (2021)
- [AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://mlsanity.com/paper/2302.13007/) (2023)
- [Data Augmentation for Intent Classification with Off-the-shelf Large Language Models](https://mlsanity.com/paper/2204.01959/) (2022)
- [PULSAR: Pre-training with Extracted Healthcare Terms for Summarising Patients' Problems and Data Augmentation with Black-box Large Language Models](https://mlsanity.com/paper/2306.02754/) (2023)

## Built on by (most cited first, 2 of 2)

- [LLMCARE: early detection of cognitive impairment via transformer models enhanced by LLM-generated synthetic data](https://mlsanity.com/paper/2508.10027/) (2025)
- [A Scoping Review of Synthetic Data Generation by Language Models in Biomedical Research and Application: Data Utility and Quality Perspectives](https://mlsanity.com/paper/2506.16594/) (2025)

## Abstract (arXiv)

An important issue impacting healthcare is a lack of available experts. Machine learning (ML) models could resolve this by aiding in diagnosing patients. However, creating datasets large enough to train these models is expensive. We evaluated large language models (LLMs) for data creation. Using Autism Spectrum Disorders (ASD), we prompted ChatGPT and GPT-Premium to generate 4,200 synthetic observations to augment existing medical data. Our goal is to label behaviors corresponding to autism criteria and improve model accuracy with synthetic training data. We used a BERT classifier pre-trained on biomedical literature to assess differences in performance between models. A random sample (N=140) from the LLM-generated data was evaluated by a clinician and found to contain 83% correct example-label pairs. Augmenting data increased recall by 13% but decreased precision by 16%, correlating with higher quality and lower accuracy across pairs. Future work will analyze how different synthetic data traits affect ML outcomes.