# Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation

arXiv: https://arxiv.org/abs/2101.06030 · HTML: https://mlsanity.com/paper/2101.06030/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2021.

## Built on by (most cited first, 23 of 23)

- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Standardizing the Measurement of Text Diversity: A Tool and a Comparative Analysis of Scores](https://mlsanity.com/paper/2403.00553/) (2024)
- [RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressions](https://mlsanity.com/paper/2302.09466/) (2023)
- [ChatGPT to Replace Crowdsourcing of Paraphrases for Intent Classification: Higher Diversity and Comparable Model Robustness](https://mlsanity.com/paper/2305.12947/) (2023)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [CloChat: Understanding How People Customize, Interact, and Experience Personas in Large Language Models](https://mlsanity.com/paper/2402.15265/) (2024)
- [Effects of diversity incentives on sample diversity and downstream model performance in LLM-based text augmentation](https://mlsanity.com/paper/2401.06643/) (2024)
- [Interpretable Directed Diversity: Leveraging Model Explanations for Iterative Crowd Ideation](https://mlsanity.com/paper/2109.10149/) (2021)
- [Prompting a Large Language Model to Generate Diverse Motivational Messages: A Comparison with Human-Written Messages](https://mlsanity.com/paper/2308.13479/) (2023)
- [IRIS: Interpretable Rubric-Informed Segmentation for Action Quality Assessment](https://mlsanity.com/paper/2303.09097/) (2023)
- [BARE: Leveraging Base Language Models for Few-Shot Synthetic Data Generation](https://mlsanity.com/paper/2502.01697/) (2025)
- [Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols](https://mlsanity.com/paper/2501.13284/) (2025)
- [Uncertainty Quantification for Retrieval-Augmented Reasoning](https://mlsanity.com/paper/2510.11483/) (2025)
- [SynQuE: Estimating Synthetic Dataset Quality Without Annotations](https://mlsanity.com/paper/2511.03928/) (2025)
- [Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models](https://mlsanity.com/paper/2501.00828/) (2025)
- [Privacy-Preserving Synthetic Review Generation with Diverse Writing Styles Using LLMs](https://mlsanity.com/paper/2507.18055/) (2025)
- [Polite But Boring? Trade-offs Between Engagement and Psychological Reactance to Chatbot Feedback Styles](https://mlsanity.com/paper/2601.20683/) (2026)
- [emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity](https://mlsanity.com/paper/2607.19848/) (2026)
- [Measuring Similarity in Causal Graphs: A Framework for Semantic and Structural Analysis](https://mlsanity.com/paper/2503.11046/) (2025)
- [PersonaTrace: Synthesizing Realistic Digital Footprints with LLM Agents](https://mlsanity.com/paper/2603.11955/) (2026)
- [DyMT-ESB: Dynamic Multi-Turn Evaluation of Social Bias in User-LLM Interactions](https://mlsanity.com/paper/2609.18649/) (2026)

## Abstract (arXiv)

Crowdsourcing can collect many diverse ideas by prompting ideators individually, but this can generate redundant ideas. Prior methods reduce redundancy by presenting peers' ideas or peer-proposed prompts, but these require much human coordination. We introduce Directed Diversity, an automatic prompt selection approach that leverages language model embedding distances to maximize diversity. Ideators can be directed towards diverse prompts and away from prior ideas, thus improving their collective creativity. Since there are diverse metrics of diversity, we present a Diversity Prompting Evaluation Framework consolidating metrics from several research disciplines to analyze along the ideation chain - prompt selection, prompt creativity, prompt-ideation mediation, and ideation creativity. Using this framework, we evaluated Directed Diversity in a series of a simulation study and four user studies for the use case of crowdsourcing motivational messages to encourage physical activity. We show that automated diverse prompting can variously improve collective creativity across many nuanced metrics of diversity.