# Sparse Mean Field Load Balancing in Large Localized Queueing Systems

arXiv: https://arxiv.org/abs/2312.12973 · HTML: https://mlsanity.com/paper/2312.12973/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms](https://mlsanity.com/paper/1911.10635/) (2019)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [Multi-agent Reinforcement Learning for Networked System Control](https://mlsanity.com/paper/2004.01339/) (2020)
- [Multi-Agent Reinforcement Learning in Stochastic Networked Systems](https://mlsanity.com/paper/2006.06555/) (2020)
- [Learning Graphon Mean Field Games and Approximate Nash Equilibria](https://mlsanity.com/paper/2112.01280/) (2021)
- [Configuring Random Graph Models with Fixed Degree Sequences](https://mlsanity.com/paper/1608.00607/) (2016)
- [Scalable Planning and Learning for Multiagent POMDPs: Extended Version](https://mlsanity.com/paper/1404.1140/) (2014)
- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Learning Sparse Graphon Mean Field Games](https://mlsanity.com/paper/2209.03880/) (2022)
- [Graphon Mean Field Games and the GMFG Equations](https://mlsanity.com/paper/2008.10216/) (2020)
- [Bellman equation and viscosity solutions for mean-field stochastic control problem](https://mlsanity.com/paper/1512.07866/) (2015)
- [Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.04808/) (2022)
- [Asymptotically Optimal Load Balancing in Large-scale Heterogeneous Systems with Multiple Dispatchers](https://mlsanity.com/paper/2002.08908/) (2020)
- [Algorithms for Mapping Parallel Processes onto Grid and Torus Architectures](https://mlsanity.com/paper/1411.0921/) (2014)
- [Learning Mean-Field Control for Delayed Information Load Balancing in Large Queuing Systems](https://mlsanity.com/paper/2208.04777/) (2022)
- [Asymptotically Optimal Load Balancing Topologies](https://mlsanity.com/paper/1707.05866/) (2017)

## Abstract (arXiv)

Scalable load balancing algorithms are of great interest in cloud networks and data centers, necessitating the use of tractable techniques to compute optimal load balancing policies for good performance. However, most existing scalable techniques, especially asymptotically scaling methods based on mean field theory, have not been able to model large queueing networks with strong locality. Meanwhile, general multi-agent reinforcement learning techniques can be hard to scale and usually lack a theoretical foundation. In this work, we address this challenge by leveraging recent advances in sparse mean field theory to learn a near-optimal load balancing policy in sparsely connected queueing networks in a tractable manner, which may be preferable to global approaches in terms of wireless communication overhead. Importantly, we obtain a general load balancing framework for a large class of sparse bounded-degree wireless topologies. By formulating a novel mean field control problem in the context of graphs with bounded degree, we reduce the otherwise difficult multi-agent problem to a single-agent problem. Theoretically, the approach is justified by approximation guarantees. Empirically, the proposed methodology performs well on several realistic and scalable wireless network topologies as compared to a number of well-known load balancing heuristics and existing scalable multi-agent reinforcement learning methods.