# Multi-Sample Prompting and Actor-Critic Prompt Optimization for Diverse Synthetic Data Generation

arXiv: https://arxiv.org/abs/2506.21138 · HTML: https://mlsanity.com/paper/2506.21138/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 15 of its 21 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [LLM preference optimization methods](https://mlsanity.com/alignment/llm-preference-optimization-methods/)
- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Batch Prompting: Efficient Inference with Large Language Model APIs](https://mlsanity.com/paper/2301.08721/) (2023)
- [PACE: Improving Prompt with Actor-Critic Editing for Large Language Model](https://mlsanity.com/paper/2308.10088/) (2023)
- [Lessons from the Use of Natural Language Inference (NLI) in Requirements Engineering Tasks](https://mlsanity.com/paper/2405.05135/) (2024)
- [Which AI Technique Is Better to Classify Requirements? An Experiment with SVM, LSTM, and ChatGPT](https://mlsanity.com/paper/2311.11547/) (2023)
- [Replication in Requirements Engineering: the NLP for RE Case](https://mlsanity.com/paper/2304.10265/) (2023)
- [Dealing with Data for RE: Mitigating Challenges while using NLP and Generative AI](https://mlsanity.com/paper/2402.16977/) (2024)
- [Automatic Prompt Optimization Techniques: Exploring the Potential for Synthetic Data Generation](https://mlsanity.com/paper/2502.03078/) (2025)
- [Synthline: A Product Line Approach for Synthetic Requirements Engineering Data Generation using Large Language Models](https://mlsanity.com/paper/2505.03265/) (2025)

## Abstract (arXiv)

High-quality labeled datasets are fundamental for training and evaluating machine learning models, yet domains such as healthcare and Requirements Engineering (RE) face persistent barriers due to data scarcity, privacy constraints, or proprietary restrictions. While Large Language Models (LLMs) offer a promising avenue for Synthetic Data Generation (SDG), LLM-generated data tends to be repetitive and low in diversity, reducing its effectiveness for downstream tasks. Two approaches show potential for addressing this limitation: (1) multi-sample prompting, which generates multiple samples per prompt to reduce repetition, and (2) Prompt with Actor-Critic Editing (PACE), which iteratively refines prompts to maximize diversity. We integrate both mechanisms into Synthline, a Feature Model-based configurable synthetic data generator, and assess their effects on diversity and downstream utility across four RE classification tasks. Multi-sample prompting consistently improves both diversity and utility, with F1-score gains of 6 to 43.8 percentage points. PACE-based prompt optimization consistently improves lexical diversity but produces task-dependent utility effects, revealing the risks of optimizing for diversity alone. Most notably, synthetic data can match or surpass human-authored data for tasks where real labeled data is limited, with improvements of up to 15.4 percentage points in F1-score.