# Multi-Agent Reinforcement Learning for Pragmatic Communication and Control

arXiv: https://arxiv.org/abs/2302.14399 · HTML: https://mlsanity.com/paper/2302.14399/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 45 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Learning to Schedule Communication in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/1902.01554/) (2019)
- [Age of Information: An Introduction and Survey](https://mlsanity.com/paper/2007.08564/) (2020)
- [Age-of-Information vs. Value-of-Information Scheduling for Cellular Networked Control Systems](https://mlsanity.com/paper/1903.05356/) (2019)
- [Urgency of Information for Context-Aware Timely Status Updates in Remote Control Systems](https://mlsanity.com/paper/2002.07987/) (2020)
- [An Adaptive Broadcasting Strategy for Efficient Dynamic Mapping in Vehicular Networks](https://mlsanity.com/paper/1910.07322/) (2019)
- [Networked control systems in the presence of scheduling protocols and communication delays](https://mlsanity.com/paper/1408.0147/) (2014)

## Built on by (most cited first, 3 of 3)

- [Effective Communication with Dynamic Feature Compression](https://mlsanity.com/paper/2401.16236/) (2024)
- [Push- and Pull-based Effective Communication in Cyber-Physical Systems](https://mlsanity.com/paper/2401.10921/) (2024)
- [Almost Sure Convergence of Networked Policy Gradient over Time-Varying Networks in Markov Potential Games](https://mlsanity.com/paper/2410.20075/) (2024)

## Abstract (arXiv)

The automation of factories and manufacturing processes has been accelerating over the past few years, boosted by the Industry 4.0 paradigm, including diverse scenarios with mobile, flexible agents. Efficient coordination between mobile robots requires reliable wireless transmission in highly dynamic environments, often with strict timing requirements. Goal-oriented communication is a possible solution for this problem: communication decisions should be optimized for the target control task, providing the information that is most relevant to decide which action to take. From the control perspective, networked control design takes the communication impairments into account in its optmization of physical actions. In this work, we propose a joint design that combines goal-oriented communication and networked control into a single optimization model, an extension of a multiagent POMDP which we call Cyber-Physical POMDP (CP-POMDP). The model is flexible enough to represent several swarm and cooperative scenarios, and we illustrate its potential with two simple reference scenarios with a single agent and a set of supporting sensors. Joint training of the communication and control systems can significantly improve the overall performance, particularly if communication is severely constrained, and can even lead to implicit coordination of communication actions.