mlsanityData to 14 Oct 2026

Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services

arXiv 2508.16037 ↗First posted 22 Aug 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Federated learning (FL) in multi-service provider (SP) ecosystems is fundamentally hampered by non-cooperative dynamics, where privacy constraints and competing interests preclude the centralized optimization of multi-SP communication and computation resources. In this paper, we introduce PAC-MCoFL, a game-theoretic multi-agent reinforcement learning (MARL) framework where SPs act as agents to jointly optimize client assignment, adaptive quantization, and resource allocation. Within the framework, we integrate Pareto Actor-Critic (PAC) principles with expectile regression, enabling agents to conjecture optimal joint policies to achieve Pareto-optimal equilibria while modeling heterogeneous risk profiles. To manage the high-dimensional action space, we devise a ternary Cartesian decomposition (TCAD) mechanism that facilitates fine-grained control. Further, we develop PAC-MCoFL-p, a scalable variant featuring a parameterized conjecture generator that substantially reduces computational complexity with a provably bounded error. Alongside theoretical convergence guarantees, our framework's superiority is validated through extensive simulations -- PAC-MCoFL achieves approximately 5.8% and 4.2% improvements in total reward and hypervolume indicator (HVI), respectively, over the latest MARL solutions. The results also demonstrate that our method can more effectively balance individual SP and system performance in scaled deployments and under diverse data heterogeneity.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 60 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

21 of 60 references.

References · 21

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  6. 2020Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Taskscited by 212
  7. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  8. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 65
  9. 2021Adaptive Quantization of Model Updates for Communication-Efficient Federated Learningcited by 63
  10. 2021Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devicescited by 56
  11. 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 31
  12. 2021SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learningcited by 26
9 more
  1. 2022Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networkingcited by 18
  2. 2022Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Designcited by 15
  3. 2022Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learningcited by 14
  4. 2020Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networkscited by 11
  5. 2021Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networkscited by 10
  6. 2023AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategycited by 7
  7. 2024Toward Finding Strong Pareto Optimal Policies in Multi-Agent Reinforcement Learningcited by 3
  8. 2023Communication-Efficient Soft Actor-Critic Policy Collaboration via Regulated Segment Mixturecited by 2
  9. 2021NeuSaver: Neural Adaptive Power Consumption Optimization for Mobile Video Streamingcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026FedCritic-MIMO: Communication-Efficient Serverless Federated Critic Learning for Massive-MIMO Resource Control in Open and Disaggregated 6G RANscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Survey of Multi Agent Reinforcement Learning: Federated Learning and Cooperative and Noncooperative Decentralized Regimes
  2. 2024Single-Loop Federated Actor-Critic across Heterogeneous Environments
  3. 2024Towards Fast Rates for Federated and Multi-Task Reinforcement Learning
  4. 2025A Service-Oriented Adaptive Hierarchical Incentive Mechanism for Federated Learning
  5. 2025FedHPD: Heterogeneous Federated Reinforcement Learning via Policy Distillation
  6. 2025Small-Scale-Fading-Aware Resource Allocation in Wireless Federated Learning
  7. 2026Agentic Federated Learning: The Future of Distributed Training Orchestration
  8. 2026FedQHD: Closed-Form Function-Space Federated Reinforcement Learning