# Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mind

arXiv: https://arxiv.org/abs/2408.12022 · HTML: https://mlsanity.com/paper/2408.12022/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 75 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 22 of 22)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Efficient Guided Generation for Large Language Models](https://mlsanity.com/paper/2307.09702/) (2023)
- [FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions](https://mlsanity.com/paper/2310.15421/) (2023)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Models](https://mlsanity.com/paper/2305.14763/) (2023)
- [From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought](https://mlsanity.com/paper/2306.12672/) (2023)
- [MMToM-QA: Multimodal Theory of Mind Question Answering](https://mlsanity.com/paper/2401.08743/) (2024)
- [Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs](https://mlsanity.com/paper/2306.03081/) (2023)
- [Meaning without reference in large language models](https://mlsanity.com/paper/2208.02957/) (2022)
- [Online Bayesian Goal Inference for Boundedly-Rational Planning Agents](https://mlsanity.com/paper/2006.07532/) (2020)
- [Turning large language models into cognitive models](https://mlsanity.com/paper/2306.03917/) (2023)
- [Word meaning in minds and machines](https://mlsanity.com/paper/2008.01766/) (2020)
- [Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning](https://mlsanity.com/paper/2402.17930/) (2024)
- [The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs](https://mlsanity.com/paper/2306.14325/) (2023)
- [Inferring the Goals of Communicating Agents from Actions and Instructions](https://mlsanity.com/paper/2306.16207/) (2023)
- [GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment](https://mlsanity.com/paper/2403.11075/) (2024)
- [A Gentle Introduction to Epistemic Planning: The DEL Approach](https://mlsanity.com/paper/1703.02192/) (2017)
- [Solving the Baby Intuitions Benchmark with a Hierarchically Bayesian Theory of Mind](https://mlsanity.com/paper/2208.02914/) (2022)
- [Identifying concept libraries from language about object structure](https://mlsanity.com/paper/2205.05666/) (2022)
- [Grounding Language about Belief in a Bayesian Theory-of-Mind](https://mlsanity.com/paper/2402.10416/) (2024)
- [Infinite Ends from Finite Samples: Open-Ended Goal Inference as Top-Down Bayesian Filtering of Bottom-Up Proposals](https://mlsanity.com/paper/2407.16770/) (2024)

## Built on by (most cited first, 6 of 6)

- [Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Models](https://mlsanity.com/paper/2502.11881/) (2025)
- [Modeling Open-World Cognition as On-Demand Synthesis of Probabilistic Models](https://mlsanity.com/paper/2507.12547/) (2025)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)
- [Belief Attribution as Mental Explanation: The Role of Accuracy, Informativity, and Causality](https://mlsanity.com/paper/2505.19376/) (2025)
- [What's in the Box? Reasoning about Unseen Objects from Multimodal Cues](https://mlsanity.com/paper/2506.14212/) (2025)
- [Realist and Pluralist Conceptions of Intelligence and Their Implications on AI Research](https://mlsanity.com/paper/2511.15282/) (2025)

## Abstract (arXiv)

How do people understand and evaluate claims about others' beliefs, even though these beliefs cannot be directly observed? In this paper, we introduce a cognitive model of epistemic language interpretation, grounded in Bayesian inferences about other agents' goals, beliefs, and intentions: a language-augmented Bayesian theory-of-mind (LaBToM). By translating natural language into an epistemic ``language-of-thought'' with grammar-constrained LLM decoding, then evaluating these translations against the inferences produced by inverting a generative model of rational action and perception, LaBToM captures graded plausibility judgments of epistemic claims. We validate our model in an experiment where participants watch an agent navigate a maze to find keys hidden in boxes needed to reach their goal, then rate sentences about the agent's beliefs. In contrast with multimodal LLMs (GPT-4o, Gemini Pro) and ablated models, our model correlates highly with human judgments for a wide range of expressions, including modal language, uncertainty expressions, knowledge claims, likelihood comparisons, and attributions of false belief.