mlsanityData to 14 Oct 2026

Forcing Diffuse Distributions out of Language Models

arXiv 2404.10859 ↗First posted 16 Apr 2024cs.CL cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

Despite being trained specifically to follow user instructions, today's instructiontuned language models perform poorly when instructed to produce random outputs. For example, when prompted to pick a number uniformly between one and ten Llama-2-13B-chat disproportionately favors the number five, and when tasked with picking a first name at random, Mistral-7B-Instruct chooses Avery 40 times more often than we would expect based on the U.S. population. When these language models are used for real-world tasks where diversity of outputs is crucial, such as language model assisted dataset construction, their inability to produce diffuse distributions over valid choices is a major hurdle. In this work, we propose a fine-tuning method that encourages language models to output distributions that are diffuse over valid outcomes. The methods we introduce generalize across a variety of tasks and distributions and make large language models practical for synthetic dataset generation with little human intervention.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

23 of 31 references.

References · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  8. 2018Hierarchical Neural Story Generationcited by 1,193
  9. 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 503
  11. 2022Unnatural Instructions: Tuning Language Models with (Almost) No Human Laborcited by 397
  12. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
11 more
  1. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 296
  2. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 211
  3. 2022Generating Training Data with Language Models: Towards Zero-Shot Language Understandingcited by 199
  4. 2021Generating Datasets with Pretrained Language Modelscited by 193
  5. 2020Evaluating the Evaluation of Diversity in Natural Language Generationcited by 129
  6. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 121
  7. 2020Trading Off Diversity and Quality in Natural Language Generationcited by 96
  8. 2022ProGen: Progressive Zero-shot Dataset Generation via In-context Feedbackcited by 67
  9. 2022RelationPrompt: Leveraging Prompts to Generate Synthetic Data for Zero-Shot Relation Triplet Extractioncited by 60
  10. 2016A Simple, Fast Diverse Decoding Algorithm for Neural Generationcited by 58
  11. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 21; 2026: 19.

48 papers.

Sort:

By area:

  1. 2024PersonaGym: Evaluating Persona Agents and LLMscited by 97
  2. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 80
  3. 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 66
  4. 2025Vision Language Models are Biasedcited by 66
  5. 2025Diverse Preference Optimizationcited by 52
  6. 2025The Best Instruction-Tuning Data are Those That Fitcited by 52
  7. 2024Benchmarking Language Model Creativity: A Case Study on Code Generationcited by 40
  8. 2025Weight Ensembling Improves Reasoning in Language Modelscited by 38
  9. 2024GenQA: Generating Millions of Instructions from a Handful of Promptscited by 35
  10. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  11. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  12. 2025Position: Model Collapse Does Not Mean What You Thinkcited by 33
36 more
  1. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 29
  2. 2025Creative Preference Optimizationcited by 25
  3. 2024Preference Optimization with Multi-Sample Comparisonscited by 16
  4. 2024Towards Fair RAG: On the Impact of Fair Ranking in Retrieval-Augmented Generationcited by 15
  5. 2025G2: Guided Generation for Enhanced Output Diversity in LLMscited by 14
  6. 2025Conversational User-AI Intervention: A Study on Prompt Rewriting for Improved LLM Response Generationcited by 10
  7. 2024Guiding and Diversifying LLM-Based Story Generation via Answer Set Programmingcited by 9
  8. 2025Always Tell Me The Odds: Fine-grained Conditional Probability Estimationcited by 9
  9. 2025Measuring Scalar Constructs in Social Science with LLMscited by 9
  10. 2026The Illusion of Stochasticity in LLMscited by 9
  11. 2025Distilled Pretraining: A modern lens of Data, In-Context Learning and Test-Time Scalingcited by 7
  12. 2026CREATE: Testing LLMs for Associative Creativitycited by 7
  13. 2025B-score: Detecting biases in large language models using response historycited by 6
  14. 2026Unlocking LLM Creativity in Science through Analogical Reasoningcited by 6
  15. 2024Mitigating Paraphrase Attacks on Machine-Text Detectors via Paraphrase Inversioncited by 5
  16. 2025Temporal Self-Rewarding Language Models: Decoupling Chosen-Rejected via Past-Futurecited by 5
  17. 2026Annotations Mitigate Post-Training Mode Collapsecited by 5
  18. 2025Geometry of Knowledge Allows Extending Diversity Boundaries of Large Language Modelscited by 3
  19. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Modelscited by 3
  20. 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Modelscited by 3
  21. 2025Who Gets the Kidney? Human-AI Alignment, Indecision, and Moral Valuescited by 2
  22. 2026The Homogenization Problem in LLMs: Towards Meaningful Diversity in AI Safetycited by 2
  23. 2026No Single Best Model for Diversity: Learning a Router for Sample Diversitycited by 2
  24. 2026Probabilistic Calibration Is a Trainable Capability in Language Modelscited by 2
  25. 2025Evolve to Inspire: Novelty Search for Diverse Image Generationcited by 1
  26. 2025Auto-SPT: Automating Semantic Preserving Transformations for Codecited by 1
  27. 2026Plausibility as Commonsense Reasoning: Humans Succeed, Large Language Models Do notcited by 1
  28. 2026Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMscited by 1
  29. 2026Anchorless Diversification for Parallel LLM Ideationcited by 1
  30. 2026AI Coding Agents in Social Science: Methodologically Diverse, Empirically Consistent, Interpretively Vulnerablecited by 1
  31. 2026When the Next Step Is Not One Step: Distribution-Aware Execution Modeling for Concurrent Go Programscited by 0
  32. 2026Improving LLMs via Validator-to-Generator Alignmentcited by 0
  33. 2026Deep and shallow biases in language modelscited by 0
  34. 2026Investigating Human--AI Discrepancies via Multiple-Solution Problemscited by 0
  35. 2026Sample What You Say: Aligning Language Models to Sample the Distributions They Statecited by 0
  36. 2026Gacha Decoding: Eliciting Diverse Generations Through Instruction Followingcited by 0