# Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2210.13942 · HTML: https://mlsanity.com/paper/2210.13942/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 25 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://mlsanity.com/paper/2201.07207/) (2022)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [The NetHack Learning Environment](https://mlsanity.com/paper/2006.13760/) (2020)
- [Goal-Conditioned Reinforcement Learning: Problems and Solutions](https://mlsanity.com/paper/2201.08299/) (2022)
- [ROMA: Multi-Agent Reinforcement Learning with Emergent Roles](https://mlsanity.com/paper/2003.08039/) (2020)
- [Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text](https://mlsanity.com/paper/2005.09382/) (2020)
- [Grounded Language Learning Fast and Slow](https://mlsanity.com/paper/2009.01719/) (2020)
- [Environmental drivers of systematicity and generalization in a situated agent](https://mlsanity.com/paper/1910.00571/) (2019)
- [Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.07393/) (2021)
- [Learning to Map Natural Language Instructions to Physical Quadcopter Control using Simulated Flight](https://mlsanity.com/paper/1910.09664/) (2019)
- [Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery](https://mlsanity.com/paper/1912.03558/) (2019)
- [Learning to Win by Reading Manuals in a Monte-Carlo Framework](https://mlsanity.com/paper/1401.5390/) (2014)
- [Guiding Policies with Language via Meta-Learning](https://mlsanity.com/paper/1811.07882/) (2018)
- [MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffer](https://mlsanity.com/paper/2206.10607/) (2022)
- [RTFM: Generalising to Novel Environment Dynamics via Reading](https://mlsanity.com/paper/1910.08210/) (2019)
- [Representation Learning for Grounded Spatial Reasoning](https://mlsanity.com/paper/1707.03938/) (2017)
- [Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learning](https://mlsanity.com/paper/2106.01404/) (2021)

## Built on by (most cited first, 2 of 2)

- [Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.10865/) (2023)
- [Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectives](https://mlsanity.com/paper/2506.09656/) (2025)

## Abstract (arXiv)

We investigate the use of natural language to drive the generalization of policies in multi-agent settings. Unlike single-agent settings, the generalization of policies should also consider the influence of other agents. Besides, with the increasing number of entities in multi-agent settings, more agent-entity interactions are needed for language grounding, and the enormous search space could impede the learning process. Moreover, given a simple general instruction,e.g., beating all enemies, agents are required to decompose it into multiple subgoals and figure out the right one to focus on. Inspired by previous work, we try to address these issues at the entity level and propose a novel framework for language grounding in multi-agent reinforcement learning, entity divider (EnDi). EnDi enables agents to independently learn subgoal division at the entity level and act in the environment based on the associated entities. The subgoal division is regularized by opponent modeling to avoid subgoal conflicts and promote coordinated strategies. Empirically, EnDi demonstrates the strong generalization ability to unseen games with new dynamics and expresses the superiority over existing methods.