# Towards Collaborative Plan Acquisition through Theory of Mind Modeling in Situated Dialogue

arXiv: https://arxiv.org/abs/2305.11271 · HTML: https://mlsanity.com/paper/2305.11271/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 61 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Machine Theory of Mind](https://mlsanity.com/paper/1802.07740/) (2018)
- [Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems](https://mlsanity.com/paper/1709.08071/) (2017)
- [Evaluating Theory of Mind in Question Answering](https://mlsanity.com/paper/1808.09352/) (2018)
- [MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks](https://mlsanity.com/paper/2109.06275/) (2021)
- [Few-shot Language Coordination by Modeling Theory of Mind](https://mlsanity.com/paper/2107.05697/) (2021)
- [Active Goal Recognition](https://mlsanity.com/paper/1909.11173/) (2019)
- [LSTM-Based Goal Recognition in Latent Space](https://mlsanity.com/paper/1808.05249/) (2018)

## Built on by (most cited first, 9 of 9)

- [Towards A Holistic Landscape of Situated Theory of Mind in Large Language Models](https://mlsanity.com/paper/2310.19619/) (2023)
- [Theory of Mind in Large Language Models: Assessment and Enhancement](https://mlsanity.com/paper/2505.00026/) (2025)
- [The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design](https://mlsanity.com/paper/2406.17949/) (2024)
- [Limits of Theory of Mind Modelling in Dialogue-Based Collaborative Plan Acquisition](https://mlsanity.com/paper/2405.12621/) (2024)
- [ToM-SSI: Evaluating Theory of Mind in Situated Social Interactions](https://mlsanity.com/paper/2509.05066/) (2025)
- [The Yokai Learning Environment: Tracking Beliefs Over Space and Time](https://mlsanity.com/paper/2508.12480/) (2025)
- [DialToM: A Theory of Mind Benchmark for Forecasting State-Driven Dialogue Trajectories](https://mlsanity.com/paper/2604.20443/) (2026)
- [ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback](https://mlsanity.com/paper/2509.05091/) (2025)
- [Human-Modeling in Sequential Decision-Making: An Analysis through the Lens of Human-Aware AI](https://mlsanity.com/paper/2405.07773/) (2024)

## Abstract (arXiv)

Collaborative tasks often begin with partial task knowledge and incomplete initial plans from each partner. To complete these tasks, agents need to engage in situated communication with their partners and coordinate their partial plans towards a complete plan to achieve a joint task goal. While such collaboration seems effortless in a human-human team, it is highly challenging for human-AI collaboration. To address this limitation, this paper takes a step towards collaborative plan acquisition, where humans and agents strive to learn and communicate with each other to acquire a complete plan for joint tasks. Specifically, we formulate a novel problem for agents to predict the missing task knowledge for themselves and for their partners based on rich perceptual and dialogue history. We extend a situated dialogue benchmark for symmetric collaborative tasks in a 3D blocks world and investigate computational strategies for plan acquisition. Our empirical results suggest that predicting the partner's missing knowledge is a more viable approach than predicting one's own. We show that explicit modeling of the partner's dialogue moves and mental states produces improved and more stable results than without. These results provide insight for future AI agents that can predict what knowledge their partner is missing and, therefore, can proactively communicate such information to help their partner acquire such missing knowledge toward a common understanding of joint tasks.