# Generative Monoculture in Large Language Models

arXiv: https://arxiv.org/abs/2407.02209 · HTML: https://mlsanity.com/paper/2407.02209/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 50 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 30 of 33)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://mlsanity.com/paper/2309.01219/) (2023)
- [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2305.11747/) (2023)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://mlsanity.com/paper/2404.09932/) (2024)
- [Do Users Write More Insecure Code with AI Assistants?](https://mlsanity.com/paper/2211.03622/) (2022)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [A Roadmap to Pluralistic Alignment](https://mlsanity.com/paper/2402.05070/) (2024)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Algorithmic Monoculture and Social Welfare](https://mlsanity.com/paper/2101.05853/) (2021)
- [Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?](https://mlsanity.com/paper/2211.13972/) (2022)

## Built on by (most cited first, 22 of 22)

- [Correlated Errors in Large Language Models](https://mlsanity.com/paper/2506.07962/) (2025)
- [Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Dataset](https://mlsanity.com/paper/2507.09650/) (2025)
- [Position: Model Collapse Does Not Mean What You Think](https://mlsanity.com/paper/2503.03150/) (2025)
- [LLM Probability Concentration: How Alignment Shrinks the Generative Horizon](https://mlsanity.com/paper/2506.17871/) (2025)
- [Value Drifts: Tracing Value Alignment During LLM Post-Training](https://mlsanity.com/paper/2510.26707/) (2025)
- [Simultaneous Reward Distillation and Preference Learning: Get You a Language Model Who Can Do Both](https://mlsanity.com/paper/2410.08458/) (2024)
- [Are LLMs becoming similarly creative? Evidence from three years of models](https://mlsanity.com/paper/2608.19437/) (2026)
- [The Variance Paradox: How AI Reduces Diversity but Increases Novelty](https://mlsanity.com/paper/2508.19264/) (2025)
- [What LLMs Think When You Don't Tell Them What to Think About?](https://mlsanity.com/paper/2602.01689/) (2026)
- [Incentives shape how humans co-create with generative AI](https://mlsanity.com/paper/2604.03529/) (2026)
- [Design and Evaluation of Multi-Agent AI Oracle Systems for Prediction Market Resolution](https://mlsanity.com/paper/2605.30802/) (2026)
- [Hitting a Moving Target: Test-Time Adaptation for AI Text Detection under Continual Distribution Shift](https://mlsanity.com/paper/2606.25152/) (2026)
- [Market Games for Generative Models: Equilibria, Welfare, and Strategic Entry](https://mlsanity.com/paper/2602.17787/) (2026)
- [LLM Jaggedness Unlocks Scientific Creativity](https://mlsanity.com/paper/2605.10574/) (2026)
- [Evaluating Pluralism in LLMs through Latent Perspectives](https://mlsanity.com/paper/2606.13254/) (2026)
- [The Hitchhiker's Guide to Monoculture: AI Homogenizes Syntax, Not (Necessarily) Semantics](https://mlsanity.com/paper/2607.13077/) (2026)
- [When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models](https://mlsanity.com/paper/2608.19230/) (2026)
- [Aligned but Flattened: Analyzing the Trade-off between Cultural Alignment and Diversity in LLMs](https://mlsanity.com/paper/2609.00565/) (2026)
- [The Oligarch Barely Steers Model Collapse in Multi-Model Ecosystems](https://mlsanity.com/paper/2609.11146/) (2026)
- [ANIMASK: What the Model Contributes to Role Play in Simulated Story Worlds](https://mlsanity.com/paper/2609.16667/) (2026)
- [Monocultural Biases: Correlated biases in large language models lead to unequal systemic exclusion rates in hiring](https://mlsanity.com/paper/2609.22169/) (2026)
- [Gacha Decoding: Eliciting Diverse Generations Through Instruction Following](https://mlsanity.com/paper/2610.01382/) (2026)

## Abstract (arXiv)

We introduce em generative monoculture, a behavior observed in large language models (LLMs) characterized by a significant narrowing of model output diversity relative to available training data for a given task: for example, generating only positive book reviews for books with a mixed reception. While in some cases, generative monoculture enhances performance (e.g., LLMs more often produce efficient code), the dangers are exacerbated in others (e.g., LLMs refuse to share diverse opinions). As LLMs are increasingly used in high-impact settings such as education and web search, careful maintenance of LLM output diversity is essential to ensure a variety of facts and perspectives are preserved over time. We experimentally demonstrate the prevalence of generative monoculture through analysis of book review and code generation tasks, and find that simple countermeasures such as altering sampling or prompting strategies are insufficient to mitigate the behavior. Moreover, our results suggest that the root causes of generative monoculture are likely embedded within the LLM's alignment processes, suggesting a need for developing fine-tuning paradigms that preserve or promote diversity.