# Multi-Agent Reinforcement Learning for Network Load Balancing in Data Center

arXiv: https://arxiv.org/abs/2201.11727 · HTML: https://mlsanity.com/paper/2201.11727/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 27 Jan 2022.

## Builds on (most cited first, 13 of 13)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [Learning Scheduling Algorithms for Data Processing Clusters](https://mlsanity.com/paper/1810.01963/) (2018)
- [Qatten: A General Framework for Cooperative Multiagent Reinforcement Learning](https://mlsanity.com/paper/2002.03939/) (2020)
- [Microservices: yesterday, today, and tomorrow](https://mlsanity.com/paper/1606.04036/) (2016)
- [MVFST-RL: An Asynchronous RL Framework for Congestion Control with Delayed Actions](https://mlsanity.com/paper/1910.04054/) (2019)
- [In-network Congestion-aware Load Balancing at Transport Layer](https://mlsanity.com/paper/1811.09731/) (2018)
- [Distributed Dispatching in the Parallel Server Model](https://mlsanity.com/paper/2008.00793/) (2020)
- [Spotlight: Scalable Transport Layer Load Balancing for Data Center Networks](https://mlsanity.com/paper/1806.08455/) (2018)
- [Reinforced Workload Distribution Fairness](https://mlsanity.com/paper/2111.00008/) (2021)

## Built on by (most cited first, 2 of 2)

- [Graph Convolutional Reinforcement Learning for Collaborative Queuing Agents](https://mlsanity.com/paper/2205.12009/) (2022)
- [Reinforcement Learning: From Algorithms To Foundation Models](https://mlsanity.com/paper/2607.17560/) (2026)

## Abstract (arXiv)

This paper presents the network load balancing problem, a challenging real-world task for multi-agent reinforcement learning (MARL) methods. Traditional heuristic solutions like Weighted-Cost Multi-Path (WCMP) and Local Shortest Queue (LSQ) are less flexible to the changing workload distributions and arrival rates, with a poor balance among multiple load balancers. The cooperative network load balancing task is formulated as a Dec-POMDP problem, which naturally induces the MARL methods. To bridge the reality gap for applying learning-based methods, all methods are directly trained and evaluated on an emulation system from moderate-to large-scale. Experiments on realistic testbeds show that the independent and "selfish" load balancing strategies are not necessarily the globally optimal ones, while the proposed MARL solution has a superior performance over different realistic settings. Additionally, the potential difficulties of MARL methods for network load balancing are analysed, which helps to draw the attention of the learning and network communities to such challenges.