# What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models

arXiv: https://arxiv.org/abs/2510.04226 · HTML: https://mlsanity.com/paper/2510.04226/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 5 Oct 2025.

## Builds on (most cited first, 28 of 28)

- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [FreshLLMs: Refreshing Large Language Models with Search Engine Augmentation](https://mlsanity.com/paper/2310.03214/) (2023)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Homogenization Effects of Large Language Models on Human Creative Ideation](https://mlsanity.com/paper/2402.01536/) (2024)
- [NoveltyBench: Evaluating Language Models for Humanlike Diversity](https://mlsanity.com/paper/2504.05228/) (2025)
- [How much do language models memorize?](https://mlsanity.com/paper/2505.24832/) (2025)
- [Are Large Language Models Consistent over Value-laden Questions?](https://mlsanity.com/paper/2407.02996/) (2024)
- [One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity](https://mlsanity.com/paper/2411.04427/) (2024)
- [Echoes in AI: Quantifying lack of plot diversity in LLM outputs](https://mlsanity.com/paper/2501.00273/) (2024)
- [Detection and Measurement of Syntactic Templates in Generated Text](https://mlsanity.com/paper/2407.00211/) (2024)
- [We're Different, We're the Same: Creative Homogeneity Across LLMs](https://mlsanity.com/paper/2501.19361/) (2025)
- [Benchmarking Linguistic Diversity of Large Language Models](https://mlsanity.com/paper/2412.10271/) (2024)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [AI and the Problem of Knowledge Collapse](https://mlsanity.com/paper/2404.03502/) (2024)
- [IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance](https://mlsanity.com/paper/2502.08395/) (2025)
- [Multilingual Prompting for Improving LLM Generation Diversity](https://mlsanity.com/paper/2505.15229/) (2025)
- [How algorithmic popularity bias hinders or promotes quality](https://mlsanity.com/paper/1707.00574/) (2017)
- [Large Language Models show both individual and collective creativity comparable to humans](https://mlsanity.com/paper/2412.03151/) (2024)
- [The Lock-in Hypothesis: Stagnation by Algorithm](https://mlsanity.com/paper/2506.06166/) (2025)
- [Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models](https://mlsanity.com/paper/2504.05325/) (2025)
- [Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essays](https://mlsanity.com/paper/2503.20062/) (2025)
- [Unstructured Evidence Attribution for Long Context Query Focused Summarization](https://mlsanity.com/paper/2502.14409/) (2025)
- [Whose Name Comes Up? Auditing LLM-Based Scholar Recommendations](https://mlsanity.com/paper/2506.00074/) (2025)
- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)

## Built on by (most cited first, 14 of 14)

- [Where does output diversity collapse in post-training?](https://mlsanity.com/paper/2604.16027/) (2026)
- [Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation](https://mlsanity.com/paper/2604.18005/) (2026)
- [Why Slop Matters](https://mlsanity.com/paper/2601.06060/) (2025)
- [Markovian Generation Chains in Large Language Models](https://mlsanity.com/paper/2603.11228/) (2026)
- [The One-Word Census: Answer-Choice Conformity Across 105 Language Models](https://mlsanity.com/paper/2607.12796/) (2026)
- [Probabilistic Calibration Is a Trainable Capability in Language Models](https://mlsanity.com/paper/2605.11845/) (2026)
- [Selecting Language Models for Social Science: Start Small, Start Open, and Validate](https://mlsanity.com/paper/2601.10926/) (2026)
- [Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling](https://mlsanity.com/paper/2603.18074/) (2026)
- [A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing](https://mlsanity.com/paper/2606.05330/) (2026)
- [Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion](https://mlsanity.com/paper/2606.09603/) (2026)
- [More Is Not More: What Matters for Diversity in LLM Opinions?](https://mlsanity.com/paper/2607.20429/) (2026)
- [Linguistic Monoculture in LLM-Assisted Language Use](https://mlsanity.com/paper/2607.27134/) (2026)
- [When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models](https://mlsanity.com/paper/2608.19230/) (2026)
- [Spicing up Genetic Netlist Generation with LLMs](https://mlsanity.com/paper/2608.23317/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used as primary knowledge sources, yet their epistemic diversity - defined as the diversity of real-world claims in their outputs - has never been measured. Low epistemic diversity would pose a risk of knowledge collapse as homogeneous LLMs mediate a shrinking in the range of accessible information over time. The dominant paradigm is that overall LLM diversity is low, but this is always with respect to a single point in time, with no reference baseline or consideration for variation across countries. We address this gap in knowledge by performing the first systematic study of epistemic diversity in LLMs across time and cultural context, testing 27 LLMs on 155 topics covering 12 countries, resulting in 1.7M responses and 70M individual claims. We find that epistemic diversity has increased substantially over the past three years, a positive counter to recent diversity pessimism. However, despite progress, we find that every system is less diverse than a search baseline. This gap is not uniform: RAG can improve diversity, while large models are counterintuitively less diverse than smaller ones. Moreover, LLM parametric knowledge systematically reflects English over local-language knowledge for country specific topics. Together, these results demonstrate that while progress on epistemic diversity is tangible, it is insufficient and unevenly distributed.