mlsanityData to 14 Oct 2026

What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models

arXiv 2510.04226 ↗First posted 5 Oct 2025cs.CL cs.AI cs.CYcited by 14

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are increasingly used as primary knowledge sources, yet their epistemic diversity - defined as the diversity of real-world claims in their outputs - has never been measured. Low epistemic diversity would pose a risk of knowledge collapse as homogeneous LLMs mediate a shrinking in the range of accessible information over time. The dominant paradigm is that overall LLM diversity is low, but this is always with respect to a single point in time, with no reference baseline or consideration for variation across countries. We address this gap in knowledge by performing the first systematic study of epistemic diversity in LLMs across time and cultural context, testing 27 LLMs on 155 topics covering 12 countries, resulting in 1.7M responses and 70M individual claims. We find that epistemic diversity has increased substantially over the past three years, a positive counter to recent diversity pessimism. However, despite progress, we find that every system is less diverse than a search baseline. This gap is not uniform: RAG can improve diversity, while large models are counterintuitively less diverse than smaller ones. Moreover, LLM parametric knowledge systematically reflects English over local-language knowledge for country specific topics. Together, these results demonstrate that while progress on epistemic diversity is tangible, it is insufficient and unevenly distributed.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 55 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

28 of 55 references.

References · 28

  1. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  3. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  4. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
  5. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 232
  6. 2023Does Writing with Language Models Reduce Content Diversity?cited by 192
  7. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 148
  8. 2024Homogenization Effects of Large Language Models on Human Creative Ideationcited by 127
  9. 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 66
  10. 2025How much do language models memorize?cited by 61
  11. 2024Are Large Language Models Consistent over Value-laden Questions?cited by 58
  12. 2024One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversitycited by 58
16 more
  1. 2024Echoes in AI: Quantifying lack of plot diversity in LLM outputscited by 48
  2. 2024Detection and Measurement of Syntactic Templates in Generated Textcited by 44
  3. 2025We're Different, We're the Same: Creative Homogeneity Across LLMscited by 42
  4. 2024Benchmarking Linguistic Diversity of Large Language Modelscited by 38
  5. 2025The Homogenizing Effect of Large Language Models on Human Expression and Thoughtcited by 35
  6. 2024AI and the Problem of Knowledge Collapsecited by 34
  7. 2025IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistancecited by 26
  8. 2025Multilingual Prompting for Improving LLM Generation Diversitycited by 25
  9. 2017How algorithmic popularity bias hinders or promotes qualitycited by 20
  10. 2024Large Language Models show both individual and collective creativity comparable to humanscited by 11
  11. 2025The Lock-in Hypothesis: Stagnation by Algorithmcited by 11
  12. 2025Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Modelscited by 7
  13. 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
  14. 2025Unstructured Evidence Attribution for Long Context Query Focused Summarizationcited by 5
  15. 2025Whose Name Comes Up? Auditing LLM-Based Scholar Recommendationscited by 5
  16. 2025A survey of diversity quantification in natural language processing: The why, what, where and howcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 13.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026On Epistemic Diversity in Large Language Models
  2. 2025A Survey of Scientific Large Language Models: From Data Foundations to Agent Frontiers
  3. 2025Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities
  4. 2026Where's Waldo? Query-language Preference under Cross-lingual Knowledge Disparities
  5. 2025Language-Specific Latent Process Hinders Cross-Lingual Performance
  6. 2025Rethinking Cross-lingual Gaps from a Statistical Viewpoint
  7. 2025Epistemic diversity across language models mitigates knowledge collapse
  8. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity