# Pareto Actor-Critic for Communication and Computation Co-Optimization in Non-Cooperative Federated Learning Services

arXiv: https://arxiv.org/abs/2508.16037 · HTML: https://mlsanity.com/paper/2508.16037/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multi-agent reinforcement learning, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 22 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [Multi-agent reinforcement learning](https://mlsanity.com/deeprl/multi-agent-reinforcement-learning/)

## Builds on (most cited first, 21 of 21)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Offline Reinforcement Learning with Implicit Q-Learning](https://mlsanity.com/paper/2110.06169/) (2021)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [Benchmarking Multi-Agent Deep Reinforcement Learning Algorithms in Cooperative Tasks](https://mlsanity.com/paper/2006.07869/) (2020)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective](https://mlsanity.com/paper/2105.05246/) (2021)
- [Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning](https://mlsanity.com/paper/2102.04487/) (2021)
- [Federated Dropout -- A Simple Approach for Enabling Federated Learning on Resource Constrained Devices](https://mlsanity.com/paper/2109.15258/) (2021)
- [A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning](https://mlsanity.com/paper/2201.02932/) (2022)
- [SHAQ: Incorporating Shapley Value Theory into Multi-Agent Q-Learning](https://mlsanity.com/paper/2105.15013/) (2021)
- [Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networking](https://mlsanity.com/paper/2205.13054/) (2022)
- [Green, Quantized Federated Learning over Wireless Networks: An Energy-Efficient Design](https://mlsanity.com/paper/2207.09387/) (2022)
- [Pareto Actor-Critic for Equilibrium Selection in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.14344/) (2022)
- [Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks](https://mlsanity.com/paper/2011.12469/) (2020)
- [Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks](https://mlsanity.com/paper/2101.03627/) (2021)
- [AQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy](https://mlsanity.com/paper/2308.00258/) (2023)
- [Toward Finding Strong Pareto Optimal Policies in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2410.19372/) (2024)
- [Communication-Efficient Soft Actor-Critic Policy Collaboration via Regulated Segment Mixture](https://mlsanity.com/paper/2312.10123/) (2023)
- [NeuSaver: Neural Adaptive Power Consumption Optimization for Mobile Video Streaming](https://mlsanity.com/paper/2107.07127/) (2021)

## Built on by (most cited first, 1 of 1)

- [FedCritic-MIMO: Communication-Efficient Serverless Federated Critic Learning for Massive-MIMO Resource Control in Open and Disaggregated 6G RANs](https://mlsanity.com/paper/2608.03852/) (2026)

## Abstract (arXiv)

Federated learning (FL) in multi-service provider (SP) ecosystems is fundamentally hampered by non-cooperative dynamics, where privacy constraints and competing interests preclude the centralized optimization of multi-SP communication and computation resources. In this paper, we introduce PAC-MCoFL, a game-theoretic multi-agent reinforcement learning (MARL) framework where SPs act as agents to jointly optimize client assignment, adaptive quantization, and resource allocation. Within the framework, we integrate Pareto Actor-Critic (PAC) principles with expectile regression, enabling agents to conjecture optimal joint policies to achieve Pareto-optimal equilibria while modeling heterogeneous risk profiles. To manage the high-dimensional action space, we devise a ternary Cartesian decomposition (TCAD) mechanism that facilitates fine-grained control. Further, we develop PAC-MCoFL-p, a scalable variant featuring a parameterized conjecture generator that substantially reduces computational complexity with a provably bounded error. Alongside theoretical convergence guarantees, our framework's superiority is validated through extensive simulations -- PAC-MCoFL achieves approximately 5.8% and 4.2% improvements in total reward and hypervolume indicator (HVI), respectively, over the latest MARL solutions. The results also demonstrate that our method can more effectively balance individual SP and system performance in scaled deployments and under diverse data heterogeneity.