# Fully Distributed Fog Load Balancing with Multi-Agent Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.12236 · HTML: https://mlsanity.com/paper/2405.12236/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 39 references that are in the map.
- First posted on arXiv 15 May 2024.

## Builds on (most cited first, 8 of 8)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [YAFS: A simulator for IoT scenarios in fog computing](https://mlsanity.com/paper/1902.01091/) (2019)
- [Blockchain as privacy and security solution for smart environments: A Survey](https://mlsanity.com/paper/2203.08901/) (2022)
- [Resilience and Load Balancing in Fog Networks: A Multi-Criteria Decision Analysis Approach](https://mlsanity.com/paper/2210.13385/) (2022)
- [Privacy-Aware Load Balancing in Fog Networks: A Reinforcement Learning Approach](https://mlsanity.com/paper/2301.09497/) (2023)
- [A Hybrid Communication Approach for Metadata Exchange in Geo-Distributed Fog Environments](https://mlsanity.com/paper/2305.13385/) (2023)
- [Lifelong Learning for Fog Load Balancing: A Transfer Learning Approach](https://mlsanity.com/paper/2310.05187/) (2023)

## Built on by (most cited first, 1 of 1)

- [Achieving Trustworthy Real-Time Decision Support Systems with Low-Latency Interpretable AI Models](https://mlsanity.com/paper/2506.20018/) (2025)

## Abstract (arXiv)

Real-time Internet of Things (IoT) applications require real-time support to handle the ever-growing demand for computing resources to process IoT workloads. Fog Computing provides high availability of such resources in a distributed manner. However, these resources must be efficiently managed to distribute unpredictable traffic demands among heterogeneous Fog resources. This paper proposes a fully distributed load-balancing solution with Multi-Agent Reinforcement Learning (MARL) that intelligently distributes IoT workloads to optimize the waiting time while providing fair resource utilization in the Fog network. These agents use transfer learning for life-long self-adaptation to dynamic changes in the environment. By leveraging distributed decision-making, MARL agents effectively minimize the waiting time compared to a single centralized agent solution and other baselines, enhancing end-to-end execution delay. Besides performance gain, a fully distributed solution allows for a global-scale implementation where agents can work independently in small collaboration regions, leveraging nearby local resources. Furthermore, we analyze the impact of a realistic frequency to observe the state of the environment, unlike the unrealistic common assumption in the literature of having observations readily available in real-time for every required action. The findings highlight the trade-off between realism and performance using an interval-based Gossip-based multi-casting protocol against assuming real-time observation availability for every generated workload.