# Cooperative Multi-Agent Learning for Navigation via Structured State Abstraction

arXiv: https://arxiv.org/abs/2306.11336 · HTML: https://mlsanity.com/paper/2306.11336/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016)
- [A Survey and Critique of Multiagent Deep Reinforcement Learning](https://mlsanity.com/paper/1810.05587/) (2018)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [Is a Good Representation Sufficient for Sample Efficient Reinforcement Learning?](https://mlsanity.com/paper/1910.03016/) (2019)
- [On the Pitfalls of Measuring Emergent Communication](https://mlsanity.com/paper/1903.05168/) (2019)
- [A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks](https://mlsanity.com/paper/2007.04979/) (2020)
- [Two Body Problem: Collaborative Visual Task Completion](https://mlsanity.com/paper/1904.05879/) (2019)
- [Learning to Ground Multi-Agent Communication with Autoencoders](https://mlsanity.com/paper/2110.15349/) (2021)
- [Abstraction for Deep Reinforcement Learning](https://mlsanity.com/paper/2202.05839/) (2022)
- [Learning to cooperate: Emergent communication in multi-agent navigation](https://mlsanity.com/paper/2004.01097/) (2020)
- [Towards Joint Learning of Optimal MAC Signaling and Wireless Channel Access](https://mlsanity.com/paper/2007.09948/) (2020)
- [Collaborative Visual Navigation](https://mlsanity.com/paper/2107.01151/) (2021)
- [Task-Oriented Data Compression for Multi-Agent Communications Over Bit-Budgeted Channels](https://mlsanity.com/paper/2005.14220/) (2020)

## Built on by (most cited first, 1 of 1)

- [Generalising Multi-Agent Cooperation through Task-Agnostic Communication](https://mlsanity.com/paper/2403.06750/) (2024)

## Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) for navigation enables agents to cooperate to achieve their navigation goals. Using emergent communication, agents learn a communication protocol to coordinate and share information that is needed to achieve their navigation tasks. In emergent communication, symbols with no pre-specified usage rules are exchanged, in which the meaning and syntax emerge through training. Learning a navigation policy along with a communication protocol in a MARL environment is highly complex due to the huge state space to be explored. To cope with this complexity, this work proposes a novel neural network architecture, for jointly learning an adaptive state space abstraction and a communication protocol among agents participating in navigation tasks. The goal is to come up with an adaptive abstractor that significantly reduces the size of the state space to be explored, without degradation in the policy performance. Simulation results show that the proposed method reaches a better policy, in terms of achievable rewards, resulting in fewer training iterations compared to the case where raw states or fixed state abstraction are used. Moreover, it is shown that a communication protocol emerges during training which enables the agents to learn better policies within fewer training iterations.