# What Do You Think I Think? Accounting for Human Beliefs Using Second-Order Theory of Mind

arXiv: https://arxiv.org/abs/2605.12745 · HTML: https://mlsanity.com/paper/2605.12745/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 26 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 3 of 3)

- [Monte Carlo Sampling Methods for Approximating Interactive POMDPs](https://mlsanity.com/paper/1401.3455/) (2014)
- [Here's What I've Learned: Asking Questions that Reveal Reward Learning](https://mlsanity.com/paper/2107.01995/) (2021)
- [Building Second-Order Mental Models for Human-Robot Interaction](https://mlsanity.com/paper/1909.06508/) (2019)

## Abstract (arXiv)

Discrepancies between an agent's actual knowledge and what a person thinks the agent knows can hinder interactions. If an agent could detect such discrepancies, it could provide feedback to account for them and improve current and future interactions. Using the I-POMDP as a framework for a second-order Theory of Mind (ToM-2), this work endows an agent with the ability to model the evolution of a person's erroneous beliefs about an agent and the cognitive biases and heuristics (CBH) from which they arise. In doing so, the agent can detect when CBH might be at play during an interaction and adaptively generate feedback that accounts for them. An in-person user study shows how a ToM-2 learner can account for the effects of a teacher's CBH to significantly improve the informativeness of teacher actions, and subjective results suggest people find the ToM-2 learner's feedback more useful.