# Bayesian Intelligence from the Outside

arXiv: https://arxiv.org/abs/2609.14724 · HTML: https://mlsanity.com/paper/2609.14724/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 13 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting](https://mlsanity.com/paper/2310.11324/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs](https://mlsanity.com/paper/2502.08640/) (2025)
- [CogBench: a large language model walks into a psychology lab](https://mlsanity.com/paper/2402.18225/) (2024)
- [Algorithmic Collusion by Large Language Models](https://mlsanity.com/paper/2404.00806/) (2024)
- [BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief](https://mlsanity.com/paper/2109.14723/) (2021)
- [Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective](https://mlsanity.com/paper/2406.00793/) (2024)
- [Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought](https://mlsanity.com/paper/2603.05488/) (2026)
- [Incoherent Probability Judgments in Large Language Models](https://mlsanity.com/paper/2401.16646/) (2024)
- [What Are the Odds? Language Models Are Capable of Probabilistic Reasoning](https://mlsanity.com/paper/2406.12830/) (2024)
- [BASIL: Bayesian Assessment of Sycophancy in LLMs](https://mlsanity.com/paper/2508.16846/) (2025)
- [QUITE: Quantifying Uncertainty in Natural Language Text in Bayesian Reasoning Scenarios](https://mlsanity.com/paper/2410.10449/) (2024)
- [Are LLM Belief Updates Consistent with Bayes' Theorem?](https://mlsanity.com/paper/2507.17951/) (2025)
- [Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexity](https://mlsanity.com/paper/2509.07339/) (2025)
- [From Evidence to Belief: A Bayesian Epistemology Approach to Language Models](https://mlsanity.com/paper/2504.19622/) (2025)
- [Martingale Score: An Unsupervised Metric for Bayesian Rationality in LLM Reasoning](https://mlsanity.com/paper/2512.02914/) (2025)
- [LLMs are not (consistently) Bayesian: Quantifying internal (in)consistencies of LLMs' probabilistic beliefs](https://mlsanity.com/paper/2605.06915/) (2026)
- [Information Design With Large Language Models](https://mlsanity.com/paper/2509.25565/) (2025)
- [Dutch Books for Language Models](https://mlsanity.com/paper/2609.02797/) (2026)
- [A Population's Feasible Posterior Beliefs](https://mlsanity.com/paper/2202.01846/) (2022)

## Abstract (arXiv)

Inferring intelligence from observable behavior is a foundational challenge in artificial intelligence. We develop a theory of Bayesian intelligence for agents such as language models. Each prompt induces a possibly imperfect internal experiment; the agent updates a full-support prior by Bayes' rule and faithfully reports its posterior over the possible answers to the question. Repetitions draw fresh, independent outcomes from the same unobserved experiment at one fixed state. We show that the agent's behavior admits this explanation if and only if its reports are not fully contradictory, i.e., some state remains possible under every report across all prompts. Report frequencies and the sizes of positive probabilities impose no further restrictions. We further propose and characterize the behavioral implications of an intelligence order that makes the behavior of two agents consistent with one agent having access to a more informative experiment: there should exist a coupling of report distributions such that the more informative agent's report excludes every answer excluded by its counterpart. Finally, we show the difficulty of aggregating coarse reports from intelligent agents: unless the agent reports a belief about the complete state of the world, the optimal aggregation can assign arbitrary weights to states that have not been excluded. These results provide a basis for understanding when agents' behavior is intelligent and highlight the difficulty of rejecting Bayesian rationality.