# Language Model Probabilities are Not Calibrated in Numeric Contexts

arXiv: https://arxiv.org/abs/2410.16007 · HTML: https://mlsanity.com/paper/2410.16007/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2024.

## Built on by (most cited first, 18 of 18)

- [Principled Context Engineering for RAG: Statistical Guarantees via Conformal Prediction](https://mlsanity.com/paper/2511.17908/) (2025)
- [Systematic Bias in Large Language Models: Discrepant Response Patterns in Binary vs. Continuous Judgment Tasks](https://mlsanity.com/paper/2504.19445/) (2025)
- [Frontier Models Can Take Actions at Low Probabilities](https://mlsanity.com/paper/2603.02202/) (2026)
- [Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces](https://mlsanity.com/paper/2609.37470/) (2026)
- [StreetMath: Study of LLMs' Approximation Behaviors](https://mlsanity.com/paper/2510.25776/) (2025)
- [Certain but not Probable? Differentiating Certainty from Probability in LLM Token Outputs for Probabilistic Scenarios](https://mlsanity.com/paper/2511.00620/) (2025)
- [Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translation](https://mlsanity.com/paper/2606.17234/) (2026)
- [How Grounded is Wikipedia? A Study on Structured Evidential Support and Retrieval](https://mlsanity.com/paper/2506.12637/) (2025)
- [Post-Norm can Resharpen Attention](https://mlsanity.com/paper/2510.08341/) (2025)
- [HARBOR: Holistic Adaptive Risk assessment model for BehaviORal healthcare](https://mlsanity.com/paper/2512.18829/) (2025)
- [Entropy-informed Decoding: Adaptive Information-Driven Branching](https://mlsanity.com/paper/2605.09745/) (2026)
- [Evaluating Stochastic Collapse and Implicit Bias in Multimodal Large Language Models](https://mlsanity.com/paper/2606.05874/) (2026)
- [When Text and Numbers Disagree: Evidence Arbitration in Large Language Models](https://mlsanity.com/paper/2608.20116/) (2026)
- [Spook the Machine: Gamified Exploration of Human Imagination of Machine Fear](https://mlsanity.com/paper/2609.15472/) (2026)
- [FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching](https://mlsanity.com/paper/2609.35673/) (2026)
- [Referential Uncertainty in Human--AI Collaboration](https://mlsanity.com/paper/2609.39518/) (2026)
- [Evaluating LLM-Generated Preference Distributions](https://mlsanity.com/paper/2610.01000/) (2026)
- [Beyond Answer Confidence: A Controlled Audit of Self-Knowledge in a Black-Box Decision Model](https://mlsanity.com/paper/2610.01006/) (2026)

## Abstract (arXiv)

Some statements have one well-defined continuation (e.g., "the Eiffel Tower is in [Paris]"), whereas others have a natural distribution over multiple options (e.g., "the weighted coin flip was [Heads/Tails].") We argue that language model (LM) outputs should capture these natural distributions. Our work specifically tests whether LM output probabilities are calibrated to numeric information within their textual contexts. For example, if the context (the prompt) concerns two equally likely options (e.g., heads or tails for a fair coin), the LM output probabilities should also be equal. Likewise, in a context with nonuniformly likely events (e.g., rolling a pair with two dice) an LM should output proportionate probabilities. However, we find that even in simple settings, the best LMs (1) are poorly calibrated and (2) have systematic biases: artifacts like word identity, word order, and word frequency all impact calibration. For example, gpt-4o-mini often picks the first of two options presented in the prompt regardless of the options' implied likelihoods, whereas Llama-3.1-8B picks the second. Models do not allocate probability mass among valid options in a calibrated manner.