# VOYAGER: A Training Free Approach for Generating Diverse Datasets using LLMs

arXiv: https://arxiv.org/abs/2512.12072 · HTML: https://mlsanity.com/paper/2512.12072/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 12 Dec 2025.

## Builds on (most cited first, 21 of 21)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://mlsanity.com/paper/1610.02424/) (2016)
- [Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement](https://mlsanity.com/paper/1903.06059/) (2019)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Truncation Sampling as Language Model Desmoothing](https://mlsanity.com/paper/2210.15191/) (2022)
- [Jointly Reinforcing Diversity and Quality in Language Model Generations](https://mlsanity.com/paper/2509.02534/) (2025)
- [Data Diversity Matters for Robust Instruction Tuning](https://mlsanity.com/paper/2311.14736/) (2023)
- [Diversity-Aware Policy Optimization for Large Language Model Reasoning](https://mlsanity.com/paper/2505.23433/) (2025)
- [Benchmarking Language Model Creativity: A Case Study on Code Generation](https://mlsanity.com/paper/2407.09007/) (2024)
- [Diversity Measurement and Subset Selection for Instruction Tuning Datasets](https://mlsanity.com/paper/2402.02318/) (2024)
- [Improving Data Efficiency via Curating LLM-Driven Rating Systems](https://mlsanity.com/paper/2410.10877/) (2024)
- [Post-training Large Language Models for Diverse High-Quality Responses](https://mlsanity.com/paper/2509.04784/) (2025)
- [Enhancing Creativity in Large Language Models through Associative Thinking Strategies](https://mlsanity.com/paper/2405.06715/) (2024)
- [Efficient Sampling for k-Determinantal Point Processes](https://mlsanity.com/paper/1509.01618/) (2015)
- [REAL Sampling: Boosting Factuality and Diversity of Open-Ended Generation via Asymptotic Entropy](https://mlsanity.com/paper/2406.07735/) (2024)

## Abstract (arXiv)

Large language models (LLMs) are increasingly being used to generate synthetic datasets for the evaluation and training of downstream models. However, prior work has noted that such generated data lacks diversity. In this paper, we propose Voyager, a novel principled approach to generate diverse datasets. Our approach is iterative and directly optimizes a mathematical quantity that optimizes the diversity of the dataset using the machinery of determinantal point processes. Furthermore, our approach is training-free, applicable to closed-source models, and scalable. In addition to providing theoretical justification for the working of our method, we also demonstrate through comprehensive experiments that Voyager significantly outperforms popular baseline approaches by providing a 1.5-3 times improvement in diversity.