What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) are increasingly used as primary knowledge sources, yet their epistemic diversity - defined as the diversity of real-world claims in their outputs - has never been measured. Low epistemic diversity would pose a risk of knowledge collapse as homogeneous LLMs mediate a shrinking in the range of accessible information over time. The dominant paradigm is that overall LLM diversity is low, but this is always with respect to a single point in time, with no reference baseline or consideration for variation across countries. We address this gap in knowledge by performing the first systematic study of epistemic diversity in LLMs across time and cultural context, testing 27 LLMs on 155 topics covering 12 countries, resulting in 1.7M responses and 70M individual claims. We find that epistemic diversity has increased substantially over the past three years, a positive counter to recent diversity pessimism. However, despite progress, we find that every system is less diverse than a search baseline. This gap is not uniform: RAG can improve diversity, while large models are counterintuitively less diverse than smaller ones. Moreover, LLM parametric knowledge systematically reflects English over local-language knowledge for country specific topics. Together, these results demonstrate that while progress on epistemic diversity is tangible, it is insufficient and unevenly distributed.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 5 Oct 2025.
Builds on
28 of 55 references.
References · 28
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
- 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
- 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
- 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
- 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
- 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
- 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
- 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 66
- 2025How much do language models memorize?cited by 61
- 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
- 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 58
16 more
- 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 48
- 2024Detection and Measurement of Syntactic Templates in Generated Textcited by 44
- 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 42
- 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 38
- 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
- 2024AI and the Problem of Knowledge Collapsecited by 34
- 2025IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistancecited by 26
- 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
- 2017How algorithmic popularity bias hinders or promotes qualitycited by 20
- 2024Large Language Models show both individual and collective creativity comparable to humanscited by 11
- 2025The Lock-in Hypothesis: Stagnation by Algorithmcited by 11
- 2025Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Modelscited by 7
- 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
- 2025Unstructured Evidence Attribution for Long Context Query Focused Summarizationcited by 5
- 2025Whose Name Comes Up? Auditing LLM-Based Scholar Recommendationscited by 5
- 2025A survey of diversity quantification in natural language processing: The why, what, where and howcited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 13.
14 papers.
Sort:
By area:
- 2026Where does output diversity collapse in post-training?cited by 11
- 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generationcited by 7
- 2025Why Slop Matterscited by 6
- 2026Markovian Generation Chains in Large Language Modelscited by 3
- 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Modelscited by 3
- 2026Probabilistic Calibration Is a Trainable Capability in Language Modelscited by 2
- 2026Selecting Language Models for Social Science: Start Small, Start Open, and Validatecited by 0
- 2026Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modelingcited by 0
- 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracingcited by 0
- 2026Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusioncited by 0
- 2026More Is Not More: What Matters for Diversity in LLM Opinions?cited by 0
- 2026Linguistic Monoculture in LLM-Assisted Language Usecited by 0
2 more
- 2026Spicing up Genetic Netlist Generation with LLMs24 Aug 2026
- 2026When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models3 Aug 2026
- 2026Linguistic Monoculture in LLM-Assisted Language Use29 Jul 2026
- 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Models14 Jul 2026
- 2026Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion8 Jun 2026
- 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing3 Jun 2026
- 2026Probabilistic Calibration Is a Trainable Capability in Language Models12 May 2026
- 2026More Is Not More: What Matters for Diversity in LLM Opinions?10 May 2026
- 2026Diversity Collapse in Multi-Agent LLM Systems: Structural Coupling and Collective Failure in Open-Ended Idea Generation20 Apr 2026
- 2026Where does output diversity collapse in post-training?17 Apr 2026
- 2026Adapting Technical-Service LLM Agents with Latent Logic Augmentation, Robust Noise Reduction, and Hybrid Reward Modeling18 Mar 2026
- 2026Markovian Generation Chains in Large Language Models11 Mar 2026
2 more
- 2026Selecting Language Models for Social Science: Start Small, Start Open, and Validate16 Jan 2026
- 2025Why Slop Matters23 Dec 2025
- 2026Where does output diversity collapse in post-training?cited by 11
- 2025Why Slop Matterscited by 6
- 2026Markovian Generation Chains in Large Language Modelscited by 3
- 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Modelscited by 3
- 2026Probabilistic Calibration Is a Trainable Capability in Language Modelscited by 2
- 2026Selecting Language Models for Social Science: Start Small, Start Open, and Validatecited by 0
- 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracingcited by 0
- 2026Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusioncited by 0
- 2026More Is Not More: What Matters for Diversity in LLM Opinions?cited by 0
- 2026Linguistic Monoculture in LLM-Assisted Language Usecited by 0
- 2026When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Modelscited by 0
- 2026Spicing up Genetic Netlist Generation with LLMscited by 0
- 2026Spicing up Genetic Netlist Generation with LLMs24 Aug 2026
- 2026When AI Writes, Who Gets Cited? Evidence of Citation Monoculture Across Language Models3 Aug 2026
- 2026Linguistic Monoculture in LLM-Assisted Language Use29 Jul 2026
- 2026The One-Word Census: Answer-Choice Conformity Across 105 Language Models14 Jul 2026
- 2026Automated IEP Generation from Traditional Chinese Parent-Teacher Interviews via Corpus-Grounded Feature Diffusion8 Jun 2026
- 2026A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing3 Jun 2026
- 2026Probabilistic Calibration Is a Trainable Capability in Language Models12 May 2026
- 2026More Is Not More: What Matters for Diversity in LLM Opinions?10 May 2026
- 2026Where does output diversity collapse in post-training?17 Apr 2026
- 2026Markovian Generation Chains in Large Language Models11 Mar 2026
- 2026Selecting Language Models for Social Science: Start Small, Start Open, and Validate16 Jan 2026
- 2025Why Slop Matters23 Dec 2025
Similar, not cited
Close in wording, no citation either way
- 2026On Epistemic Diversity in Large Language Models
- 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers
- 2025Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
- 2026Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities
- 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
- 2025Rethinking Cross-lingual Gaps from a Statistical Viewpoint
- 2025Epistemic diversity across language models mitigates knowledge collapse
- 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity