# D-Models and E-Models: Diversity-Stability Trade-offs in the Sampling Behavior of Large Language Models

arXiv: https://arxiv.org/abs/2601.17865 · HTML: https://mlsanity.com/paper/2601.17865/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 25 Jan 2026.

## Builds on (most cited first, 18 of 18)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [RLPR: Extrapolating RLVR to General Domains without Verifiers](https://mlsanity.com/paper/2506.18254/) (2025)
- [Base Models Beat Aligned Models at Randomness and Creativity](https://mlsanity.com/paper/2505.00047/) (2025)
- [Cycles of Thought: Measuring LLM Confidence through Stable Explanations](https://mlsanity.com/paper/2406.03441/) (2024)
- [ThinkSum: Probabilistic reasoning over sets using large language models](https://mlsanity.com/paper/2210.01293/) (2022)
- [Enough Coin Flips Can Make LLMs Act Bayesian](https://mlsanity.com/paper/2503.04722/) (2025)
- [Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation](https://mlsanity.com/paper/2404.09043/) (2024)
- [CTRLS: Chain-of-Thought Reasoning via Latent State-Transition](https://mlsanity.com/paper/2507.08182/) (2025)
- [Manipulating LLM Web Agents with Indirect Prompt Injection Attack via HTML Accessibility Tree](https://mlsanity.com/paper/2507.14799/) (2025)
- [Fourier Head: Helping Large Language Models Learn Complex Probability Distributions](https://mlsanity.com/paper/2410.22269/) (2024)
- [Unveiling Confirmation Bias in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2506.12301/) (2025)
- [B-score: Detecting biases in large language models using response history](https://mlsanity.com/paper/2505.18545/) (2025)
- [LLM-Driven Usefulness Judgment for Web Search Evaluation](https://mlsanity.com/paper/2504.14401/) (2025)

## Built on by (most cited first, 1 of 1)

- [Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs](https://mlsanity.com/paper/2607.25196/) (2026)

## Abstract (arXiv)

The predictive probability of the next token (P_token) in large language models (LLMs) is inextricably linked to the probability of relevance for the next piece of information, the purchase probability of the next product, and the execution probability of the next action-all of which fall under the scope of the task-level target distribution (P_task). While LLMs are known to generate samples that approximate real-world distributions, whether their fine-grained sampling probabilities faithfully align with task requirements remains an open question. Through controlled distribution-sampling simulations, we uncover a striking dichotomy in LLM behavior, distinguishing two model types: D-models (e.g. Qwen-2.5), whose P_token exhibits large step-to-step variability and poor alignment with P_task; and E-models (e.g. Mistral-Small), whose P_token is more stable and better aligned with P_task. We further evaluate these two model types in downstream tasks such as code generation and recommendation, revealing systematic trade-offs between diversity and stability that shape task outcomes. Finally, we analyze the internal properties of both model families to probe their underlying mechanisms. These findings offer foundational insights into the probabilistic sampling behavior of LLMs and provide practical guidance on when to favor D- versus E-models. For web-scale applications, including recommendation, search, and conversational agents, our results inform model selection and configuration to balance diversity with reliability under real-world uncertainty, providing a better level of interpretation.