# Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communication

arXiv: https://arxiv.org/abs/2409.17348 · HTML: https://mlsanity.com/paper/2409.17348/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 29 of 29)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Learning Multiagent Communication with Backpropagation](https://mlsanity.com/paper/1605.07736/) (2016)
- [Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning](https://mlsanity.com/paper/2302.02662/) (2023)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [Theory of Mind for Multi-Agent Collaboration via Large Language Models](https://mlsanity.com/paper/2310.10701/) (2023)
- [Language Models Meet World Models: Embodied Experiences Enhance Language Models](https://mlsanity.com/paper/2305.10626/) (2023)
- [Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks](https://mlsanity.com/paper/1812.09755/) (2018)
- [Using Natural Language for Reward Shaping in Reinforcement Learning](https://mlsanity.com/paper/1903.02020/) (2019)
- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017)
- [Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning](https://mlsanity.com/paper/2005.07064/) (2020)
- [Anti-efficient encoding in emergent communication](https://mlsanity.com/paper/1905.12561/) (2019)
- [Countering Language Drift via Visual Grounding](https://mlsanity.com/paper/1909.04499/) (2019)
- [Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.07393/) (2021)
- [Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals](https://mlsanity.com/paper/2302.04449/) (2023)
- [Learning to Ground Multi-Agent Communication with Autoencoders](https://mlsanity.com/paper/2110.15349/) (2021)
- [True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning](https://mlsanity.com/paper/2401.14151/) (2024)
- [Emergent Discrete Communication in Semantic Spaces](https://mlsanity.com/paper/2108.01828/) (2021)
- [Linking Emergent and Natural Languages via Corpus Transfer](https://mlsanity.com/paper/2203.13344/) (2022)
- [Deep Interpretable Models of Theory of Mind](https://mlsanity.com/paper/2104.02938/) (2021)
- [Learning Multi-Agent Communication with Contrastive Learning](https://mlsanity.com/paper/2307.01403/) (2023)
- [Interpretable Learned Emergent Communication for Human-Agent Teams](https://mlsanity.com/paper/2201.07452/) (2022)
- [Community Regularization of Visually-Grounded Dialog](https://mlsanity.com/paper/1808.04359/) (2018)
- [Grounding Natural Language Commands to StarCraft II Game States for Narration-Guided Reinforcement Learning](https://mlsanity.com/paper/1906.02671/) (2019)
- [Communication breakdown: On the low mutual intelligibility between human and neural captioning](https://mlsanity.com/paper/2210.11512/) (2022)
- [On the Role of Emergent Communication for Social Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2302.14276/) (2023)

## Built on by (most cited first, 17 of 17)

- [Reasoning on a Spectrum: Aligning LLMs to System 1 and System 2 Thinking](https://mlsanity.com/paper/2502.12470/) (2025)
- [Maestro: Learning to Collaborate via Conditional Listwise Policy Optimization for Multi-Agent LLMs](https://mlsanity.com/paper/2511.06134/) (2025)
- [The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperation](https://mlsanity.com/paper/2511.02687/) (2025)
- [The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning](https://mlsanity.com/paper/2502.15214/) (2025)
- [HiMA-Ecom: Enabling Joint Training of Hierarchical Multi-Agent E-commerce Assistants](https://mlsanity.com/paper/2506.19846/) (2025)
- [Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning](https://mlsanity.com/paper/2603.07972/) (2026)
- [IPCGRL: Language-Instructed Reinforcement Learning for Procedural Level Generation](https://mlsanity.com/paper/2503.12358/) (2025)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [From Virtual Agents to Robot Teams: A Multi-Robot Framework Evaluation in High-Stakes Healthcare Context](https://mlsanity.com/paper/2506.03546/) (2025)
- [Game-Theoretic Lens on LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2601.15047/) (2026)
- [Deception and Communication in Autonomous Multi-Agent Systems: An Experimental Study with Among Us](https://mlsanity.com/paper/2603.26635/) (2026)
- [DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention](https://mlsanity.com/paper/2602.13255/) (2026)
- [Prompting Robot Teams with Natural Language](https://mlsanity.com/paper/2509.24575/) (2025)
- [Learning to Trust: Bayesian Adaptation to Varying Suggester Reliability in Sequential Decision Making](https://mlsanity.com/paper/2511.12378/) (2025)
- [LLM-ALSO: LLM-Driven Adaptive Learning-Signal Optimization for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.29293/) (2026)
- [Emergent Language as an Approach to Conscious AI](https://mlsanity.com/paper/2606.06380/) (2026)
- [DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication](https://mlsanity.com/paper/2610.02161/) (2026)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) methods have shown promise in enabling agents to learn a shared communication protocol from scratch and accomplish challenging team tasks. However, the learned language is usually not interpretable to humans or other agents not co-trained together, limiting its applicability in ad-hoc teamwork scenarios. In this work, we propose a novel computational pipeline that aligns the communication space between MARL agents with an embedding space of human natural language by grounding agent communications on synthetic data generated by embodied Large Language Models (LLMs) in interactive teamwork scenarios. Our results demonstrate that introducing language grounding not only maintains task performance but also accelerates the emergence of communication. Furthermore, the learned communication protocols exhibit zero-shot generalization capabilities in ad-hoc teamwork scenarios with unseen teammates and novel task states. This work presents a significant step toward enabling effective communication and collaboration between artificial agents and humans in real-world teamwork settings.