# Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind

arXiv: https://arxiv.org/abs/2106.13249 · HTML: https://mlsanity.com/paper/2106.13249/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 29 references that are in the map.
- First posted on arXiv 24 Jun 2021.

## Builds on (most cited first, 5 of 5)

- [Online Bayesian Goal Inference for Boundedly-Rational Planning Agents](https://mlsanity.com/paper/2006.07532/) (2020)
- [Learning the Preferences of Ignorant, Inconsistent Agents](https://mlsanity.com/paper/1512.05832/) (2015)
- [Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior](https://mlsanity.com/paper/1805.08010/) (2018)
- [LESS is More: Rethinking Probabilistic Models of Human Behavior](https://mlsanity.com/paper/2001.04465/) (2020)
- [Getting to Know One Another: Calibrating Intent, Capabilities and Trust for Human-Robot Collaboration](https://mlsanity.com/paper/2008.00699/) (2020)

## Built on by (most cited first, 19 of 19)

- [Building Machines that Learn and Think with People](https://mlsanity.com/paper/2408.03943/) (2024)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [Beyond Preferences in AI Alignment](https://mlsanity.com/paper/2408.16984/) (2024)
- [Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning](https://mlsanity.com/paper/2402.17930/) (2024)
- [The Effect of Modeling Human Rationality Level on Learning Rewards from Multiple Feedback Types](https://mlsanity.com/paper/2208.10687/) (2022)
- [The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs](https://mlsanity.com/paper/2306.14325/) (2023)
- [Cognitive science as a source of forward and inverse models of human decisions for robotics and control](https://mlsanity.com/paper/2109.00127/) (2021)
- [Inferring the Goals of Communicating Agents from Actions and Instructions](https://mlsanity.com/paper/2306.16207/) (2023)
- [Solving the Baby Intuitions Benchmark with a Hierarchically Bayesian Theory of Mind](https://mlsanity.com/paper/2208.02914/) (2022)
- [Why we need biased AI -- How including cognitive and ethical machine biases can enhance AI systems](https://mlsanity.com/paper/2203.09911/) (2022)
- [Infinite Ends from Finite Samples: Open-Ended Goal Inference as Top-Down Bayesian Filtering of Bottom-Up Proposals](https://mlsanity.com/paper/2407.16770/) (2024)
- [Language-Informed Synthesis of Rational Agent Models for Grounded Theory-of-Mind Reasoning On-The-Fly](https://mlsanity.com/paper/2506.16755/) (2025)
- [Concept Alignment as a Prerequisite for Value Alignment](https://mlsanity.com/paper/2310.20059/) (2023)
- [Learning Goal-Conditioned Representations for Language Reward Models](https://mlsanity.com/paper/2407.13887/) (2024)
- [Beyond Words: Evaluating and Bridging Epistemic Divergence in User-Agent Interaction via Theory of Mind](https://mlsanity.com/paper/2602.13832/) (2026)
- [Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind](https://mlsanity.com/paper/2411.07003/) (2024)
- [Belief Attribution as Mental Explanation: The Role of Accuracy, Informativity, and Causality](https://mlsanity.com/paper/2505.19376/) (2025)
- [Rational Inverse Reasoning: Few-Shot Imitation by Inferring Intent through Planning](https://mlsanity.com/paper/2508.08983/) (2025)
- [Propose to Learn, Learn to Propose: Evaluability-Aware Assistance under Bounded Rationality](https://mlsanity.com/paper/2609.02242/) (2026)

## Abstract (arXiv)

When inferring the goals that others are trying to achieve, people intuitively understand that others might make mistakes along the way. This is crucial for activities such as teaching, offering assistance, and deciding between blame or forgiveness. However, Bayesian models of theory of mind have generally not accounted for these mistakes, instead modeling agents as mostly optimal in achieving their goals. As a result, they are unable to explain phenomena like locking oneself out of one's house, or losing a game of chess. Here, we extend the Bayesian Theory of Mind framework to model boundedly rational agents who may have mistaken goals, plans, and actions. We formalize this by modeling agents as probabilistic programs, where goals may be confused with semantically similar states, plans may be misguided due to resource-bounded planning, and actions may be unintended due to execution errors. We present experiments eliciting human goal inferences in two domains: (i) a gridworld puzzle with gems locked behind doors, and (ii) a block-stacking domain. Our model better explains human inferences than alternatives, while generalizing across domains. These findings indicate the importance of modeling others as bounded agents, in order to account for the full richness of human intuitive psychology.