# Joint Optimization of Training and Inference in Federated Edge Learning via Constrained Multi-Objective Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.25916 · HTML: https://mlsanity.com/paper/2605.25916/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [Federated Learning in Mobile Edge Networks: A Comprehensive Survey](https://mlsanity.com/paper/1909.11875/) (2019)
- [Federated Learning for Internet of Things: A Comprehensive Survey](https://mlsanity.com/paper/2104.07914/) (2021)
- [Energy Efficient Federated Learning Over Wireless Communication Networks](https://mlsanity.com/paper/1911.02417/) (2019)
- [IPO: Interior-point Policy Optimization under Constraints](https://mlsanity.com/paper/1910.09615/) (2019)
- [Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learning](https://mlsanity.com/paper/2007.07174/) (2020)
- [Scheduling for Cellular Federated Edge Learning with Importance and Channel Awareness](https://mlsanity.com/paper/2004.00490/) (2020)
- [Optimized Power Control Design for Over-the-Air Federated Edge Learning](https://mlsanity.com/paper/2106.09316/) (2021)
- [Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraints](https://mlsanity.com/paper/2106.00490/) (2021)
- [Federated Learning-Empowered AI-Generated Content in Wireless Networks](https://mlsanity.com/paper/2307.07146/) (2023)
- [Resource Allocation for Multiuser Edge Inference with Batching and Early Exiting (Extended Version)](https://mlsanity.com/paper/2204.05223/) (2022)
- [On the Local Cache Update Rules in Streaming Federated Learning](https://mlsanity.com/paper/2303.16340/) (2023)
- [Joint Age-based Client Selection and Resource Allocation for Communication-Efficient Federated Learning over NOMA Networks](https://mlsanity.com/paper/2304.08996/) (2023)
- [Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization](https://mlsanity.com/paper/2312.12863/) (2023)
- [Collaborative Ground-Space Communications via Evolutionary Multi-objective Deep Reinforcement Learning](https://mlsanity.com/paper/2404.07450/) (2024)
- [Timely Communications for Remote Inference](https://mlsanity.com/paper/2404.16281/) (2024)
- [Towards Dynamic Resource Allocation and Client Scheduling in Hierarchical Federated Learning: A Two-Phase Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2406.14910/) (2024)
- [Federated Learning with Noisy User Feedback](https://mlsanity.com/paper/2205.03092/) (2022)
- [Inference Load-Aware Orchestration for Hierarchical Federated Learning](https://mlsanity.com/paper/2407.16836/) (2024)
- [Personalized Federated Deep Reinforcement Learning for Heterogeneous Edge Content Caching Networks](https://mlsanity.com/paper/2412.12543/) (2024)
- [Age and Value of Information Optimization for Systems with Multi-Class Updates](https://mlsanity.com/paper/2408.12589/) (2024)

## Abstract (arXiv)

Federated edge learning (FEEL) has recently emerged as a promising paradigm for achieving edge intelligence (EI) via enabling collaborative model training across edge devices while protecting data privacy. In this paper, we put forth an online optimization framework that jointly manages federated training and inference on resource-constrained edge devices. We introduce a tandem-queue-inspired conversion mechanism that bridges inference requests and training data, and further incorporate both data and model freshness into the accuracy formulation to capture temporal dynamics in real-world environments. To maximize inference accuracy while minimizing latency and energy consumption, the mode selections, communication, and computation resource allocations of edge devices are jointly optimized. We formulate this optimization as a multi-objective optimization problem, which is NP-hard and further complicated by the online setting. To address these challenges, we transform the problem into a multi-objective Markov decision process (MOMDP) and develop a underlineconstrained underlinemulti-underlineobjective underlineproximal underlinepolicy underlineoptimization (C-MOPPO) algorithm. Specifically, C-MOPPO first learns a set of policies with different preferences across three objectives, then leverages constrained policy optimization to enrich the Pareto front and obtain high-quality, dense solutions. Extensive experiments demonstrate that C-MOPPO achieves well-balanced trade-offs among objectives and significantly outperforms baselines under various system configurations.