# Enhancing Robot Assistive Behaviour with Reinforcement Learning and Theory of Mind

arXiv: https://arxiv.org/abs/2411.07003 · HTML: https://mlsanity.com/paper/2411.07003/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 67 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 5 of 5)

- [Modeling the Mistakes of Boundedly Rational Agents Within a Bayesian Theory of Mind](https://mlsanity.com/paper/2106.13249/) (2021)
- [Explainable Autonomous Robots: A Survey and Perspective](https://mlsanity.com/paper/2105.02658/) (2021)
- [When Would You Trust a Robot? A Study on Trust and Theory of Mind in Human-Robot Interactions](https://mlsanity.com/paper/2101.10819/) (2021)
- [Theory of Mind Based Assistive Communication in Complex Human Robot Cooperation](https://mlsanity.com/paper/2109.01355/) (2021)
- [Explain yourself! Effects of Explanations in Human-Robot Interaction](https://mlsanity.com/paper/2204.04501/) (2022)

## Abstract (arXiv)

The adaptation to users' preferences and the ability to infer and interpret humans' beliefs and intents, which is known as the Theory of Mind (ToM), are two crucial aspects for achieving effective human-robot collaboration. Despite its importance, very few studies have investigated the impact of adaptive robots with ToM abilities. In this work, we present an exploratory comparative study to investigate how social robots equipped with ToM abilities impact users' performance and perception. We design a two-layer architecture. The Q-learning agent on the first layer learns the robot's higher-level behaviour. On the second layer, a heuristic-based ToM infers the user's intended strategy and is responsible for implementing the robot's assistance, as well as providing the motivation behind its choice. We conducted a user study in a real-world setting, involving 56 participants who interacted with either an adaptive robot capable of ToM, or with a robot lacking such abilities. Our findings suggest that participants in the ToM condition performed better, accepted the robot's assistance more often, and perceived its ability to adapt, predict and recognise their intents to a higher degree. Our preliminary insights could inform future research and pave the way for designing more complex computation architectures for adaptive behaviour with ToM capabilities.