mlsanityData to 14 Oct 2026

Scaling Synthetic Data Creation with 1,000,000,000 Personas

arXiv 2406.20094 ↗First posted 28 Jun 2024cs.CL cs.LGcited by 417

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM) to create diverse synthetic data. To fully exploit this methodology at scale, we introduce Persona Hub -- a collection of 1 billion diverse personas automatically curated from web data. These 1 billion personas (~13% of the world's total population), acting as distributed carriers of world knowledge, can tap into almost every perspective encapsulated within the LLM, thereby facilitating the creation of diverse synthetic data at scale for various scenarios. By showcasing Persona Hub's use cases in synthesizing high-quality mathematical and logical reasoning problems, instructions (i.e., user prompts), knowledge-rich texts, game NPCs and tools (functions) at scale, we demonstrate persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on LLM research and development.

Key facts. Cited by 417 papers in the map (data through 14 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 39 of its 44 references that are in the map. First posted on arXiv 28 Jun 2024.

Builds on

39 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Best Practices and Lessons Learned on Synthetic Data11 Apr

Other references · 38

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  8. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  9. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  10. 2024Yi: Open Foundation Models by 01.AIcited by 719
  11. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  12. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
26 more
  1. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 565
  2. 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
  3. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  4. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
  5. 2023Large Language Models as Tool Makerscited by 303
  6. 2023Role-Play with Large Language Modelscited by 294
  7. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 290
  8. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 251
  9. 2023Language Modeling Is Compressioncited by 225
  10. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 199
  11. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  12. 2024Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modelingcited by 161
  13. 2023MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoningcited by 156
  14. 2024A Tale of Tails: Model Collapse as a Change of Scaling Lawscited by 130
  15. 2023On the Risk of Misinformation Pollution with Large Language Modelscited by 128
  16. 2024Common 7B Language Models Already Possess Strong Math Capabilitiescited by 109
  17. 2024Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Modelscited by 99
  18. 2024LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Modelscited by 99
  19. 2023On the Possibilities of AI-Generated Text Detectioncited by 88
  20. 2023Strategic Reasoning with Language Modelscited by 64
  21. 2024Comprehensive Exploration of Synthetic Data Generation: A Surveycited by 60
  22. 2024Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoningcited by 58
  23. 2023On the steerability of large language models toward data-driven personascited by 53
  24. 2023Faithful Persona-based Conversational Dataset Generation with Large Language Modelscited by 47
  25. 2024COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuningcited by 45
  26. 2024PICLe: Eliciting Diverse Behaviors from Large Language Models with Persona In-Context Learningcited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 45; 2025: 191; 2026: 181.

417 papers.

Sort:

By area:

  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  2. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  3. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 329
  4. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
  5. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplaycited by 167
  6. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  7. 2025PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memorycited by 88
  8. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  9. 2025LLM Generated Persona is a Promise with a Catchcited by 81
  10. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 80
  11. 2024Aligning LLMs with Individual Preferences via Interactioncited by 63
  12. 2024PersonalLLM: Tailoring LLMs to Individual Preferencescited by 60
38 more
  1. 2024On the Diversity of Synthetic Data and its Impact on Training Large Language Modelscited by 59
  2. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  3. 2025LongCat-Flash Technical Reportcited by 58
  4. 2025HaluMem: Evaluating Hallucinations in Memory Systems of Agentscited by 54
  5. 2025AgentRefine: Enhancing Agent Generalization through Refinement Tuningcited by 53
  6. 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
  7. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 52
  8. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  9. 2025MAS-GPT: Training LLMs to Build LLM-based Multi-Agent Systemscited by 50
  10. 2025A Survey of Large Language Model Agents for Question Answeringcited by 48
  11. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 47
  12. 2025Scaling Laws of Synthetic Data for Language Modelscited by 46
  13. 2025AgentSynth: Scalable Task Generation for Generalist Computer-Use Agentscited by 46
  14. 2025Reasoning to Learn from Latent Thoughtscited by 45
  15. 2025Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modelingcited by 45
  16. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 45
  17. 2025The Price of Format: Diversity Collapse in LLMscited by 43
  18. 2025Cognitive Kernel-Pro: A Framework for Deep Research Agents and Agent Foundation Models Trainingcited by 42
  19. 2024A Survey on Data Synthesis and Augmentation for Large Language Modelscited by 41
  20. 2025Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoningcited by 40
  21. 2024AI-Assisted Generation of Difficult Math Questionscited by 37
  22. 2025Multi-Agent Coordination across Diverse Applications: A Surveycited by 37
  23. 2025KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Modelcited by 36
  24. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  25. 2025TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Recordscited by 34
  26. 2024AI Safety in Generative AI Large Language Models: A Surveycited by 33
  27. 2026Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environmentscited by 33
  28. 2026Tmax: A simple recipe for terminal agentscited by 33
  29. 2025Language Model Personalization via Reward Factorizationcited by 31
  30. 2025Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoningcited by 31
  31. 2025Recycling the Web: A Method to Enhance Pre-training Data Quality and Quantity for Language Modelscited by 31
  32. 2025ARE: Scaling Up Agent Environments and Evaluationscited by 30
  33. 2025OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulationcited by 28
  34. 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 27
  35. 2025PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Datacited by 27
  36. 2025ECom-Bench: Can LLM Agent Resolve Real-World E-commerce Customer Support Issues?cited by 27
  37. 2025CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulationcited by 26
  38. 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Executioncited by 26

50 most cited of 417 All 417 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignment
  2. 2025Whose Personae? Synthetic Persona Experiments in LLM Research and Pathways to Transparency
  3. 2024Agentic Society: Merging skeleton from real world and texture from Large Language Model
  4. 2024PersonaGym: Evaluating Persona Agents and LLMs
  5. 2026How to Model AI Agents as Personas?: Applying the Persona Ecosystem Playground to 41,300 Posts on Moltbook for Behavioral Insights
  6. 2024A Synthetic Dataset for Personal Attribute Inference
  7. 2025Polypersona: Persona-Grounded LLM for Synthetic Survey Responses
  8. 2025PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization