# Coordinated Reinforcement Learning for Optimizing Mobile Networks

arXiv: https://arxiv.org/abs/2109.15175 · HTML: https://mlsanity.com/paper/2109.15175/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 11 of 11)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [A Review of Cooperative Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1908.03963/) (2019)
- [Deep Coordination Graphs](https://mlsanity.com/paper/1910.00091/) (2019)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Online Antenna Tuning in Heterogeneous Cellular Networks with Deep Reinforcement Learning](https://mlsanity.com/paper/1903.06787/) (2019)
- [Self-Tuning Sectorization: Deep Reinforcement Learning Meets Broadcast Beam Optimization](https://mlsanity.com/paper/1906.06021/) (2019)
- [Off-policy Learning for Remote Electrical Tilt Optimization](https://mlsanity.com/paper/2005.10577/) (2020)
- [Utility Fair Optimisation of Antenna Tilt Angles in LTE Networks](https://mlsanity.com/paper/1310.1015/) (2013)

## Built on by (most cited first, 7 of 7)

- [Agent-Time Attention for Sparse Rewards Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.17540/) (2022)
- [Multi-agent Reinforcement Learning with Graph Q-Networks for Antenna Tuning](https://mlsanity.com/paper/2302.01199/) (2023)
- [Model Based Residual Policy Learning with Applications to Antenna Control](https://mlsanity.com/paper/2211.08796/) (2022)
- [Continual Model-based Reinforcement Learning for Data Efficient Wireless Network Optimisation](https://mlsanity.com/paper/2404.19462/) (2024)
- [Online Frequency Scheduling by Learning Parallel Actions](https://mlsanity.com/paper/2406.05041/) (2024)
- [Action Dependency Graphs for Globally Optimal Coordinated Reinforcement Learning](https://mlsanity.com/paper/2506.00797/) (2025)
- [Optimizing Reinforcement Learning Training over Digital Twin Enabled Multi-fidelity Networks](https://mlsanity.com/paper/2603.08931/) (2026)

## Abstract (arXiv)

Mobile networks are composed of many base stations and for each of them many parameters must be optimized to provide good services. Automatically and dynamically optimizing all these entities is challenging as they are sensitive to variations in the environment and can affect each other through interferences. Reinforcement learning (RL) algorithms are good candidates to automatically learn base station configuration strategies from incoming data but they are often hard to scale to many agents. In this work, we demonstrate how to use coordination graphs and reinforcement learning in a complex application involving hundreds of cooperating agents. We show how mobile networks can be modeled using coordination graphs and how network optimization problems can be solved efficiently using multi- agent reinforcement learning. The graph structure occurs naturally from expert knowledge about the network and allows to explicitly learn coordinating behaviors between the antennas through edge value functions represented by neural networks. We show empirically that coordinated reinforcement learning outperforms other methods. The use of local RL updates and parameter sharing can handle a large number of agents without sacrificing coordination which makes it well suited to optimize the ever denser networks brought by 5G and beyond.