# A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning

arXiv: https://arxiv.org/abs/2201.02932 · HTML: https://mlsanity.com/paper/2201.02932/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 9 Jan 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Towards Federated Learning at Scale: System Design](https://mlsanity.com/paper/1902.01046/) (2019)
- [Federated Multi-Task Learning](https://mlsanity.com/paper/1705.10467/) (2017)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Robust and Communication-Efficient Federated Learning from Non-IID Data](https://mlsanity.com/paper/1903.02891/) (2019)
- [Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks](https://mlsanity.com/paper/1901.06523/) (2019)
- [Adaptive Personalized Federated Learning](https://mlsanity.com/paper/2003.13461/) (2020)
- [FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization](https://mlsanity.com/paper/1909.13014/) (2019)
- [Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge](https://mlsanity.com/paper/1804.08333/) (2018)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Personalized Federated Learning with First Order Model Optimization](https://mlsanity.com/paper/2012.08565/) (2020)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [FetchSGD: Communication-Efficient Federated Learning with Sketching](https://mlsanity.com/paper/2007.07682/) (2020)
- [Learning Attentional Communication for Multi-Agent Cooperation](https://mlsanity.com/paper/1805.07733/) (2018)
- [Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning](https://mlsanity.com/paper/2105.05883/) (2021)
- [Detailed comparison of communication efficiency of split learning and federated learning](https://mlsanity.com/paper/1909.09145/) (2019)
- [Mitigating Bias in Federated Learning](https://mlsanity.com/paper/2012.02447/) (2020)
- [Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control](https://mlsanity.com/paper/1909.02682/) (2019)
- [Succinct and Robust Multi-Agent Communication With Temporal Message Control](https://mlsanity.com/paper/2010.14391/) (2020)
- [Coded Federated Learning](https://mlsanity.com/paper/2002.09574/) (2020)

## Built on by (most cited first, 30 of 31)

- [Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction](https://mlsanity.com/paper/2310.08670/) (2023)
- [SphereFed: Hyperspherical Federated Learning](https://mlsanity.com/paper/2207.09413/) (2022)
- [Stochastic Distributed Optimization under Average Second-order Similarity: Algorithms and Analysis](https://mlsanity.com/paper/2304.07504/) (2023)
- [FedZero: Leveraging Renewable Excess Energy in Federated Learning](https://mlsanity.com/paper/2305.15092/) (2023)
- [When Do Curricula Work in Federated Learning?](https://mlsanity.com/paper/2212.12712/) (2022)
- [FedGCS: A Generative Framework for Efficient Client Selection in Federated Learning via Gradient-based Optimization](https://mlsanity.com/paper/2405.06312/) (2024)
- [DLoRA: Distributed Parameter-Efficient Fine-Tuning Solution for Large Language Model](https://mlsanity.com/paper/2404.05182/) (2024)
- [Advancements in Federated Learning: Models, Methods, and Privacy](https://mlsanity.com/paper/2302.11466/) (2023)
- [Unlocking Visual Secrets: Inverting Features with Diffusion Priors for Image Reconstruction](https://mlsanity.com/paper/2412.10448/) (2024)
- [DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving](https://mlsanity.com/paper/2511.21669/) (2025)
- [Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective](https://mlsanity.com/paper/2405.20431/) (2024)
- [On the Linear Speedup of Personalized Federated Reinforcement Learning with Shared Representations](https://mlsanity.com/paper/2411.15014/) (2024)
- [Fairness in Federated Learning: Trends, Challenges, and Opportunities](https://mlsanity.com/paper/2509.00799/) (2025)
- [FLASH-RL: Federated Learning Addressing System and Static Heterogeneity using Reinforcement Learning](https://mlsanity.com/paper/2311.06917/) (2023)
- [Ranking-based Client Selection with Imitation Learning for Efficient Federated Learning](https://mlsanity.com/paper/2405.04122/) (2024)
- [Learnable Sparse Customization in Heterogeneous Edge Computing](https://mlsanity.com/paper/2412.07216/) (2024)
- [Tool-Aided Evolutionary LLM for Generative Policy Toward Efficient Resource Management in Wireless Federated Learning](https://mlsanity.com/paper/2505.11570/) (2025)
- [FedDRL: A Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning](https://mlsanity.com/paper/2307.13716/) (2023)
- [Honest Score Client Selection Scheme: Preventing Federated Learning Label Flipping Attacks in Non-IID Scenarios](https://mlsanity.com/paper/2311.05826/) (2023)
- [Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services](https://mlsanity.com/paper/2508.16037/) (2025)
- [FedHeN: Federated Learning in Heterogeneous Networks](https://mlsanity.com/paper/2207.03031/) (2022)
- [Near-optimal Differentially Private Client Selection in Federated Settings](https://mlsanity.com/paper/2310.09370/) (2023)
- [Stratify: Rethinking Federated Learning for Non-IID Data through Balanced Sampling](https://mlsanity.com/paper/2504.13462/) (2025)
- [Towards Machine Learning-based Model Predictive Control for HVAC Control in Multi-Context Buildings at Scale via Ensemble Learning](https://mlsanity.com/paper/2505.02439/) (2025)
- [Small-Scale-Fading-Aware Resource Allocation in Wireless Federated Learning](https://mlsanity.com/paper/2505.03533/) (2025)
- [Tackling Heterogeneity in Federated Learning via Variance-Reduced Boltzmann Sampling within Homogeneous Social Coalitions](https://mlsanity.com/paper/2506.02897/) (2025)
- [BIPPO: Budget-Aware Independent PPO for Energy-Efficient Federated Learning Services](https://mlsanity.com/paper/2511.08142/) (2025)
- [Federated Learning at the Forefront of Fairness: A Multifaceted Perspective](https://mlsanity.com/paper/2602.00718/) (2026)
- [Similarity-Aware Personalized Federated Learning in Heterogeneous Environments](https://mlsanity.com/paper/2609.02241/) (2026)
- [Task-oriented Framework for Communication-Efficient Federated Learning: From Isolated Optimization to Holistic Synergy](https://mlsanity.com/paper/2609.04945/) (2026)

## Abstract (arXiv)

Federated learning (FL) is a training technique that enables client devices to jointly learn a shared model by aggregating locally-computed models without exposing their raw data. While most of the existing work focuses on improving the FL model accuracy, in this paper, we focus on the improving the training efficiency, which is often a hurdle for adopting FL in real-world applications. Specifically, we design an efficient FL framework which jointly optimizes model accuracy, processing latency and communication efficiency, all of which are primary design considerations for real implementation of FL. Inspired by the recent success of Multi-Agent Reinforcement Learning (MARL) in solving complex control problems, we present textitFedMarl, an MARL-based FL framework which performs efficient run-time client selection. Experiments show that FedMarl can significantly improve model accuracy with much lower processing latency and communication cost.