# Toward a Graded Measure of Belief Stability in Large Language Models

arXiv: https://arxiv.org/abs/2609.34158 · HTML: https://mlsanity.com/paper/2609.34158/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023)
- [To Believe or Not to Believe Your LLM](https://mlsanity.com/paper/2406.02543/) (2024)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [Large Language Models Must Be Taught to Know What They Don't Know](https://mlsanity.com/paper/2406.08391/) (2024)
- [Linguistic Calibration of Long-Form Generations](https://mlsanity.com/paper/2404.00474/) (2024)
- [BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief](https://mlsanity.com/paper/2109.14723/) (2021)
- [SPUQ: Perturbation-Based Uncertainty Quantification for Large Language Models](https://mlsanity.com/paper/2403.02509/) (2024)
- [Distinguishing the Knowable from the Unknowable with Language Models](https://mlsanity.com/paper/2402.03563/) (2024)
- [Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension](https://mlsanity.com/paper/2402.18048/) (2024)
- [Firm or Fickle? Evaluating Large Language Models Consistency in Sequential Interactions](https://mlsanity.com/paper/2503.22353/) (2025)
- [The Truthfulness Spectrum Hypothesis](https://mlsanity.com/paper/2602.20273/) (2026)
- [The Trilemma of Truth in Large Language Models](https://mlsanity.com/paper/2506.23921/) (2025)
- [Certain and Uncertain Inference with Indicative Conditionals](https://mlsanity.com/paper/2207.08276/) (2022)
- [Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models](https://mlsanity.com/paper/2607.15277/) (2026)
- [Language Models Encode the Contextual Truth of Propositions](https://mlsanity.com/paper/2608.03035/) (2026)
- [Epistemic Familiarity is Associated With Belief Stability in Large Language Models](https://mlsanity.com/paper/2511.19166/) (2025)
- [Robust for the Wrong Reasons: The Representational Geometry of LLM Robustness to Science Skepticism](https://mlsanity.com/paper/2607.01951/) (2026)

## Abstract (arXiv)

Large language models (LLMs) increasingly mediate how people access and reason with information, yet factual reliability is usually evaluated one judgment at a time. We introduce graded belief stability, a relational measure of how well a belief persists within an LLM's broader belief system. Unlike individual belief probability, it asks whether support for a claim persists when that claim is considered alongside the model's other epistemic commitments. We operationalize this idea with a Direct Conditional estimator that uses internal model representations to estimate conditional belief probabilities. Across 12 LLMs and three domains, lower-stability beliefs exhibit greater mean behavioral movement under conversational challenge in 83.3% of model-domain settings after matching on individual belief probability. Graded belief stability therefore extends reliability assessment beyond how strongly an LLM supports a claim to how robustly that belief is supported within its broader system of beliefs.