# Federated Learning With Energy Harvesting Devices: An MDP Framework

arXiv: https://arxiv.org/abs/2405.10513 · HTML: https://mlsanity.com/paper/2405.10513/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Condition](https://mlsanity.com/paper/1608.04636/) (2016)
- [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Energy Efficient Federated Learning Over Wireless Communication Networks](https://mlsanity.com/paper/1911.02417/) (2019)
- [The Roadmap to 6G -- AI Empowered Wireless Networks](https://mlsanity.com/paper/1904.11686/) (2019)
- [FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Data](https://mlsanity.com/paper/2005.11418/) (2020)
- [Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges](https://mlsanity.com/paper/2009.13012/) (2020)
- [Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective](https://mlsanity.com/paper/2004.04314/) (2020)
- [Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applications](https://mlsanity.com/paper/2111.12444/) (2021)
- [Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation](https://mlsanity.com/paper/1910.13067/) (2019)
- [Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning](https://mlsanity.com/paper/2007.07174/) (2020)
- [Over-the-Air Federated Learning from Heterogeneous Data](https://mlsanity.com/paper/2009.12787/) (2020)
- [Optimized Power Control Design for Over-the-Air Federated Edge Learning](https://mlsanity.com/paper/2106.09316/) (2021)
- [Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints](https://mlsanity.com/paper/2106.00490/) (2021)
- [Convergence Analysis and System Design for Federated Learning over Wireless Networks](https://mlsanity.com/paper/2105.00872/) (2021)
- [Federated Learning over Energy Harvesting Wireless Networks](https://mlsanity.com/paper/2106.08809/) (2021)
- [On the Complexity of Value Iteration](https://mlsanity.com/paper/1807.04920/) (2018)
- [Over-the-Air Federated Learning with Energy Harvesting Devices](https://mlsanity.com/paper/2205.12869/) (2022)
- [Energy-Harvesting Distributed Machine Learning](https://mlsanity.com/paper/2102.05639/) (2021)
- [On Federated Learning with Energy Harvesting Clients](https://mlsanity.com/paper/2202.06105/) (2022)
- [Gradual Federated Learning with Simulated Annealing](https://mlsanity.com/paper/2110.05178/) (2021)
- [Interval Dominance based Structural Results for Markov Decision Process](https://mlsanity.com/paper/2203.10618/) (2022)
- [Communication Efficient ConFederated Learning: An Event-Triggered SAGA Approach](https://mlsanity.com/paper/2402.18018/) (2024)

## Built on by (most cited first, 4 of 4)

- [Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices](https://mlsanity.com/paper/2501.18298/) (2025)
- [Battery-aware Cyclic Scheduling in Energy-harvesting Federated Learning](https://mlsanity.com/paper/2504.12181/) (2025)
- [Decentralized Federated Learning With Energy Harvesting Devices](https://mlsanity.com/paper/2602.14051/) (2026)
- [Cluster-Aware Over-the-Air Federated Learning with Energy-Harvesting Devices: From Global Training to Model Personalization](https://mlsanity.com/paper/2608.01426/) (2026)

## Abstract (arXiv)

Federated learning (FL) necessitates that edge devices conduct local training and communicate with a parameter server, resulting in significant energy consumption. A key challenge in practical FL systems is the rapid depletion of battery-limited edge devices, which limits their operational lifespan and impacts learning performance. To tackle this issue, we implement energy harvesting techniques in FL systems to capture ambient energy, thereby providing continuous power to edge devices. We first establish the convergence bound for the wireless FL system with energy harvesting devices, illustrating that the convergence is affected by partial device participation and packet drops, both of which depend on the energy supply. To accelerate the convergence, we formulate a joint device scheduling and power control problem and model it as a Markov decision process (MDP). By solving this MDP, we derive the optimal transmission policy and demonstrate that it possesses a monotone structure with respect to the battery and channel states. To overcome the curse of dimensionality caused by the exponential complexity of computing the optimal policy, we propose a low-complexity algorithm, which is asymptotically optimal as the number of devices increases. Furthermore, for unknown channels and harvested energy statistics, we develop a structure-enhanced deep reinforcement learning algorithm that leverages the monotone structure of the optimal policy to improve the training performance. Finally, extensive numerical experiments on real-world datasets are presented to validate the theoretical results and corroborate the effectiveness of the proposed algorithms.