# Lost in Sampling: Assessing Lexical Reachability in LLMs via the Word Coverage Score (WCS)

arXiv: https://arxiv.org/abs/2605.27268 · HTML: https://mlsanity.com/paper/2605.27268/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 23 references that are in the map.
- First posted on arXiv 26 May 2026.

## Builds on (most cited first, 13 of 13)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Compressive Transformers for Long-Range Sequence Modelling](https://mlsanity.com/paper/1911.05507/) (2019)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [Generative artificial intelligence enhances creativity but reduces the diversity of novel content](https://mlsanity.com/paper/2312.00506/) (2023)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)](https://mlsanity.com/paper/2510.22954/) (2025)
- [Is Temperature the Creativity Parameter of Large Language Models?](https://mlsanity.com/paper/2405.00492/) (2024)
- [Evaluating the Diversity and Quality of LLM Generated Content](https://mlsanity.com/paper/2504.12522/) (2025)
- [The Price of Format: Diversity Collapse in LLMs](https://mlsanity.com/paper/2505.18949/) (2025)
- [Beware of Words: Evaluating the Lexical Diversity of Conversational LLMs using ChatGPT as Case Study](https://mlsanity.com/paper/2402.15518/) (2024)
- [Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics](https://mlsanity.com/paper/2604.11012/) (2026)
- [The Alignment Tax: Response Homogenization in Aligned LLMs and Its Implications for Uncertainty Estimation](https://mlsanity.com/paper/2603.24124/) (2026)
- [The Surprising Universality of LLM Outputs: A Real-Time Verification Primitive](https://mlsanity.com/paper/2604.25634/) (2026)

## Built on by (most cited first, 1 of 1)

- [Adaptive Nucleus Truncation for Long-Form Reasoning](https://mlsanity.com/paper/2606.13982/) (2026)

## Abstract (arXiv)

Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabularies. While previous research has focused on model knowledge and training data, we investigate the role of decoding mechanics in suppressing linguistic diversity. We introduce the Word Coverage Score (WCS), a metric that quantifies the extent to which contextually appropriate human vocabulary is mathematically pruned by standard sampling filters (e.g., Top-p, Top-k, and Min-p). Rather than assessing static knowledge, the WCS measures the lexical survival rate of low-frequency, high-information human words as a function of sampling parameters. By auditing open-weight models on human-authored corpus fragments, we identify which logical lexical choices are rendered unreachable by the decoder, even when they reside within the probability space. Our results provide quantitative evidence that industry-standard sampling defaults act as unintended censorship mechanisms, smoothing the unique textures of human expression into a homogenized discourse. The WCS offers a rigorous framework for optimizing the trade-off between text coherence and lexical richness, providing a diagnostic tool for preserving the diversity of human language in generative models.