# GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment

arXiv: https://arxiv.org/abs/2403.11075 · HTML: https://mlsanity.com/paper/2403.11075/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks](https://mlsanity.com/paper/2302.08399/) (2023)
- [Building Cooperative Embodied Agents Modularly with Large Language Models](https://mlsanity.com/paper/2307.02485/) (2023)
- [RoCo: Dialectic Multi-Robot Collaboration with Large Language Models](https://mlsanity.com/paper/2307.04738/) (2023)
- [Watch-And-Help: A Challenge for Social Perception and Human-AI Collaboration](https://mlsanity.com/paper/2010.09890/) (2020)
- [MMToM-QA: Multimodal Theory of Mind Question Answering](https://mlsanity.com/paper/2401.08743/) (2024)
- [Online Bayesian Goal Inference for Boundedly-Rational Planning Agents](https://mlsanity.com/paper/2006.07532/) (2020)
- [AGENT: A Benchmark for Core Psychological Reasoning](https://mlsanity.com/paper/2102.12321/) (2021)
- [Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planning](https://mlsanity.com/paper/2402.17930/) (2024)
- [Plan Explicability and Predictability for Robot Task Planning](https://mlsanity.com/paper/1511.08158/) (2015)
- [The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputs](https://mlsanity.com/paper/2306.14325/) (2023)
- [Inferring the Goals of Communicating Agents from Actions and Instructions](https://mlsanity.com/paper/2306.16207/) (2023)
- [Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks](https://mlsanity.com/paper/2007.12803/) (2020)
- [Expressing Robot Incapability](https://mlsanity.com/paper/1810.08167/) (2018)
- [Monte Carlo Sampling Methods for Approximating Interactive POMDPs](https://mlsanity.com/paper/1401.3455/) (2014)
- [Grounding Language about Belief in a Bayesian Theory-of-Mind](https://mlsanity.com/paper/2402.10416/) (2024)
- [Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration](https://mlsanity.com/paper/2103.04077/) (2021)

## Built on by (most cited first, 16 of 16)

- [MuMA-ToM: Multi-modal Multi-Agent Theory of Mind](https://mlsanity.com/paper/2408.12574/) (2024)
- [AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling](https://mlsanity.com/paper/2502.15676/) (2025)
- [Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration](https://mlsanity.com/paper/2412.18292/) (2024)
- [Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination](https://mlsanity.com/paper/2504.12714/) (2025)
- [Generative Multi-Agent Collaboration in Embodied AI: A Systematic Review](https://mlsanity.com/paper/2502.11518/) (2025)
- [The Overcooked Generalisation Challenge: Evaluating Cooperation with Novel Partners in Unknown Environments Using Unsupervised Environment Design](https://mlsanity.com/paper/2406.17949/) (2024)
- [Infer Human's Intentions Before Following Natural Language Instructions](https://mlsanity.com/paper/2409.18073/) (2024)
- [CAMON: Cooperative Agents for Multi-Object Navigation with LLM-based Conversations](https://mlsanity.com/paper/2407.00632/) (2024)
- [Modeling Others' Minds as Code](https://mlsanity.com/paper/2510.01272/) (2025)
- [Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mind](https://mlsanity.com/paper/2408.12022/) (2024)
- [Pragmatic Embodied Spoken Instruction Following in Human-Robot Collaboration with Theory of Mind](https://mlsanity.com/paper/2409.10849/) (2024)
- [BeliefNest: A Joint Action Simulator for Embodied Agents with Theory of Mind](https://mlsanity.com/paper/2505.12321/) (2025)
- [ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback](https://mlsanity.com/paper/2509.05091/) (2025)
- [Mosaic: Runtime-Efficient Multi-Agent Embodied Planning](https://mlsanity.com/paper/2607.09603/) (2026)
- [IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models](https://mlsanity.com/paper/2604.24002/) (2026)
- [MindZero: Learning Online Mental Reasoning With Zero Annotations](https://mlsanity.com/paper/2606.00240/) (2026)

## Abstract (arXiv)

Verbal communication plays a crucial role in human cooperation, particularly when the partners only have incomplete information about the task, environment, and each other's mental state. In this paper, we propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA). GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the parts of agents' mental states that are relevant to the goals. This approach enables an embodied assistant to reason about when and how to proactively initialize communication with humans verbally using natural language to help achieve better cooperation. We evaluate our approach against strong baselines in two challenging environments, Overcooked (a multiplayer game) and VirtualHome (a household simulator). Our experimental results demonstrate that large language models struggle with generating meaningful communication that is grounded in the social and physical context. In contrast, our approach can successfully generate concise verbal communication for the embodied assistant to effectively boost the performance of the cooperation as well as human users' perception of the assistant.