# Lifelong Learning for Fog Load Balancing: A Transfer Learning Approach

arXiv: https://arxiv.org/abs/2310.05187 · HTML: https://mlsanity.com/paper/2310.05187/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 8 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Transfer Learning in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2009.07888/) (2020)
- [Deep Reinforcement Learning Based Mode Selection and Resource Management for Green Fog Radio Access Networks](https://mlsanity.com/paper/1809.05629/) (2018)
- [YAFS: A simulator for IoT scenarios in fog computing](https://mlsanity.com/paper/1902.01091/) (2019)
- [Blockchain as privacy and security solution for smart environments: A Survey](https://mlsanity.com/paper/2203.08901/) (2022)
- [Resilience and Load Balancing in Fog Networks: A Multi-Criteria Decision Analysis Approach](https://mlsanity.com/paper/2210.13385/) (2022)
- [Privacy-Aware Load Balancing in Fog Networks: A Reinforcement Learning Approach](https://mlsanity.com/paper/2301.09497/) (2023)

## Built on by (most cited first, 1 of 1)

- [Fully Distributed Fog Load Balancing with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2405.12236/) (2024)

## Abstract (arXiv)

Fog computing emerged as a promising paradigm to address the challenges of processing and managing data generated by the Internet of Things (IoT). Load balancing (LB) plays a crucial role in Fog computing environments to optimize the overall system performance. It requires efficient resource allocation to improve resource utilization, minimize latency, and enhance the quality of service for end-users. In this work, we improve the performance of privacy-aware Reinforcement Learning (RL) agents that optimize the execution delay of IoT applications by minimizing the waiting delay. To maintain privacy, these agents optimize the waiting delay by minimizing the change in the number of queued requests in the whole system, i.e., without explicitly observing the actual number of requests that are queued in each Fog node nor observing the compute resource capabilities of those nodes. Besides improving the performance of these agents, we propose in this paper a lifelong learning framework for these agents, where lightweight inference models are used during deployment to minimize action delay and only retrained in case of significant environmental changes. To improve the performance, minimize the training cost, and adapt the agents to those changes, we explore the application of Transfer Learning (TL). TL transfers the knowledge acquired from a source domain and applies it to a target domain, enabling the reuse of learned policies and experiences. TL can be also used to pre-train the agent in simulation before fine-tuning it in the real environment; this significantly reduces failure probability compared to learning from scratch in the real environment. To our knowledge, there are no existing efforts in the literature that use TL to address lifelong learning for RL-based Fog LB; this is one of the main obstacles in deploying RL LB solutions in Fog systems.