# On-Demand Model and Client Deployment in Federated Learning with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.07175 · HTML: https://mlsanity.com/paper/2405.07175/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 12 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 19 of 19)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalization](https://mlsanity.com/paper/2107.04649/) (2021)
- [A Hybrid Approach to Privacy-Preserving Federated Learning](https://mlsanity.com/paper/1812.03224/) (2018)
- [PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models -- Federated Learning in Age of Foundation Model](https://mlsanity.com/paper/2208.11625/) (2022)
- [Jump-Start Reinforcement Learning](https://mlsanity.com/paper/2204.02372/) (2022)
- [Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective](https://mlsanity.com/paper/2004.04314/) (2020)
- [Decentralized Federated Learning: A Segmented Gossip Approach](https://mlsanity.com/paper/1908.07782/) (2019)
- [Flexible Clustered Federated Learning for Client-Level Data Distribution Shift](https://mlsanity.com/paper/2108.09749/) (2021)
- [Communication-Efficient Federated Learning with Compensated Overlap-FedAvg](https://mlsanity.com/paper/2012.06706/) (2020)
- [Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraints](https://mlsanity.com/paper/2205.06730/) (2022)
- [The ALAMO approach to machine learning](https://mlsanity.com/paper/1705.10918/) (2017)
- [Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT](https://mlsanity.com/paper/2202.03575/) (2022)
- [ON-DEMAND-FL: A Dynamic and Efficient Multi-Criteria Federated Learning Client Deployment Scheme](https://mlsanity.com/paper/2211.02906/) (2022)
- [FedMint: Intelligent Bilateral Client Selection in Federated Learning with Newcomer IoT Devices](https://mlsanity.com/paper/2211.01805/) (2022)
- [ModularFed: Leveraging Modularity in Federated Learning Frameworks](https://mlsanity.com/paper/2212.10427/) (2022)
- [Emergence of Adaptive Circadian Rhythms in Deep Reinforcement Learning](https://mlsanity.com/paper/2307.12143/) (2023)
- [Trust Driven On-Demand Scheme for Client Deployment in Federated Learning](https://mlsanity.com/paper/2405.00395/) (2024)

## Built on by (most cited first, 1 of 1)

- [Demonstrating Data-to-Knowledge Pipelines for Connecting Production Sites in the World Wide Lab](https://mlsanity.com/paper/2412.12231/) (2024)

## Abstract (arXiv)

In Federated Learning (FL), the limited accessibility of data from diverse locations and user types poses a significant challenge due to restricted user participation. Expanding client access and diversifying data enhance models by incorporating diverse perspectives, thereby enhancing adaptability. However, challenges arise in dynamic and mobile environments where certain devices may become inaccessible as FL clients, impacting data availability and client selection methods. To address this, we propose an On-Demand solution, deploying new clients using Docker Containers on-the-fly. Our On-Demand solution, employing Deep Reinforcement Learning (DRL), targets client availability and selection, while considering data shifts, and container deployment complexities. It employs an autonomous end-to-end solution for handling model deployment and client selection. The DRL strategy uses a Markov Decision Process (MDP) framework, with a Master Learner and a Joiner Learner. The designed cost functions represent the complexity of the dynamic client deployment and selection. Simulated tests show that our architecture can easily adjust to changes in the environment and respond to On-Demand requests. This underscores its ability to improve client availability, capability, accuracy, and learning efficiency, surpassing heuristic and tabular reinforcement learning solutions.