# Embedding-Driven Diversity Sampling to Improve Few-Shot Synthetic Data Generation

arXiv: https://arxiv.org/abs/2501.11199 · HTML: https://mlsanity.com/paper/2501.11199/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical synthetic data and agents, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 20 Jan 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Medical synthetic data and agents](https://mlsanity.com/synth/medical-synthetic-data-and-agents/)

## Builds on (most cited first, 14 of 14)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Publicly Available Clinical BERT Embeddings](https://mlsanity.com/paper/1904.03323/) (2019)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization](https://mlsanity.com/paper/2309.07430/) (2023)
- [A Study of Generative Large Language Model for Medical Research and Healthcare](https://mlsanity.com/paper/2305.13523/) (2023)
- [Annotation-efficient deep learning for automatic medical image segmentation](https://mlsanity.com/paper/2012.04885/) (2020)
- [Towards Conversational Diagnostic AI](https://mlsanity.com/paper/2401.05654/) (2024)
- [Zero-Shot Clinical Trial Patient Matching with LLMs](https://mlsanity.com/paper/2402.05125/) (2024)
- [Two Directions for Clinical Data Generation with Large Language Models: Data-to-Label and Label-to-Data](https://mlsanity.com/paper/2401.06774/) (2023)
- [CORAL: Expert-Curated medical Oncology Reports to Advance Language Model Inference](https://mlsanity.com/paper/2308.03853/) (2023)
- [Enhancing Clinical Documentation with Synthetic Data: Leveraging Generative Models for Improved Accuracy](https://mlsanity.com/paper/2406.06569/) (2024)

## Built on by (most cited first, 1 of 1)

- [Efficient Embedding-based Synthetic Data Generation for Complex Reasoning Tasks](https://mlsanity.com/paper/2603.22294/) (2026)

## Abstract (arXiv)

Accurate classification of clinical text often requires fine-tuning pre-trained language models, a process that is costly and time-consuming due to the need for high-quality data and expert annotators. Synthetic data generation offers an alternative, though pre-trained models may not capture the syntactic diversity of clinical notes. We propose an embedding-driven approach that uses diversity sampling from a small set of real clinical notes to guide large language models in few-shot prompting, generating synthetic text that better reflects clinical syntax. We evaluated this method using the CheXpert dataset on a classification task, comparing it to random few-shot and zero-shot approaches. Using cosine similarity and a Turing test, our approach produced synthetic notes that more closely align with real clinical text. Our pipeline reduced the data needed to reach the 0.85 AUC cutoff by 40% for AUROC and 30% for AUPRC, while augmenting models with synthetic data improved AUROC by 57% and AUPRC by 68%. Additionally, our synthetic data was 0.9 times as effective as real data, a 60% improvement in value.