# On Epistemic Diversity in Large Language Models

arXiv: https://arxiv.org/abs/2609.04835 · HTML: https://mlsanity.com/paper/2609.04835/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 29 of 29)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases](https://mlsanity.com/paper/2006.03955/) (2020)
- [Large language models that replace human participants can harmfully misportray and flatten identity groups](https://mlsanity.com/paper/2402.01908/) (2024)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [Is Temperature the Creativity Parameter of Large Language Models?](https://mlsanity.com/paper/2405.00492/) (2024)
- [Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data](https://mlsanity.com/paper/2410.13341/) (2024)
- [Evaluating the Diversity and Quality of LLM Generated Content](https://mlsanity.com/paper/2504.12522/) (2025)
- [Echoes in AI: Quantifying lack of plot diversity in LLM outputs](https://mlsanity.com/paper/2501.00273/) (2024)
- [Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Voting](https://mlsanity.com/paper/2310.16523/) (2023)
- [Benchmarking Linguistic Diversity of Large Language Models](https://mlsanity.com/paper/2412.10271/) (2024)
- [Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric](https://mlsanity.com/paper/2502.17184/) (2025)
- [Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media](https://mlsanity.com/paper/1704.01347/) (2017)
- [Geographical Erasure in Language Generation](https://mlsanity.com/paper/2310.14777/) (2023)
- [The Value of Disagreement in AI Design, Evaluation, and Alignment](https://mlsanity.com/paper/2505.07772/) (2025)
- [Growing a Tail: Increasing Output Diversity in Large Language Models](https://mlsanity.com/paper/2411.02989/) (2024)
- [SATA-BENCH: Select All That Apply Benchmark for Multiple Choice Questions](https://mlsanity.com/paper/2506.00643/) (2025)
- [One world, one opinion? The superstar effect in LLM responses](https://mlsanity.com/paper/2412.10281/) (2024)
- [Avoidance Decoding for Diverse Multi-Branch Story Generation](https://mlsanity.com/paper/2509.02170/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly used not only to retrieve information, but to answer questions, explain, teach, and support inquiry. In such settings, evaluation cannot be exhausted by accuracy or alignment alone. A system may give a correct answer while still narrowing users' access %to knowledge. to alternative valid answers, explanations, or reasoning routes. Drawing on the broader notion of epistemic diversity in philosophy and social epistemology, we formalize it in the context of LLMs as the range of valid answers, explanations, and reasoning routes that an LLM exposes to users. We argue that epistemic diversity is a useful evaluation dimension for settings where LLMs are used to support knowledge-intensive tasks. We propose a preliminary framework for conceptualizing and measuring epistemic diversity in LLMs, and operationalize it in two domains. We find that frontier LLMs often exhibit epistemic narrowness, repeatedly collapsing large valid answer spaces onto small canonical subsets. These findings suggest that LLM evaluation should move beyond accuracy-oriented paradigms and treat epistemic diversity as an important dimension of model capability.