mlsanityData to 7 Oct 2026

Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward

arXiv 2006.06626 ↗First posted 11 Jun 2020math.OC cs.AI cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

It has long been recognized that multi-agent reinforcement learning (MARL) faces significant scalability issues due to the fact that the size of the state and action spaces are exponentially large in the number of agents. In this paper, we identify a rich class of networked MARL problems where the model exhibits a local dependence structure that allows it to be solved in a scalable manner. Specifically, we propose a Scalable Actor-Critic (SAC) method that can learn a near optimal localized policy for optimizing the average reward with complexity scaling with the state-action space size of local neighborhoods, as opposed to the entire network. Our result centers around identifying and exploiting an exponential decay property that ensures the effect of agents on each other decays exponentially fast in their graph distance.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 11; 2023: 4; 2024: 10; 2025: 7; 2026: 9.

46 papers.

Sort:

By area:

  1. 2024Multi-Agent Reinforcement Learning for Autonomous Driving: A Surveycited by 30
  2. 2023A Review of Cooperation in Multi-agent Learningcited by 27
  3. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 21
  4. 2024AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Makingcited by 19
  5. 2022Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learningcited by 18
  6. 2021Causal Multi-Agent Reinforcement Learning: Review and Open Problemscited by 16
  7. 2022A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learningcited by 12
  8. 2024Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learningcited by 9
  9. 2023Mean-Field Control based Approximation of Multi-Agent Reinforcement Learning in Presence of a Non-decomposable Shared Global Statecited by 8
  10. 2024Efficient Reinforcement Learning for Global Decision Making in the Presence of Local Agents at Scalecited by 8
  11. 2022Distributed Multi-Agent Reinforcement Learning Based on Graph-Induced Local Value Functionscited by 6
  12. 2022Scalable Model-based Policy Optimization for Decentralized Networked Systemscited by 6
34 more
  1. 2024Locally Interdependent Multi-Agent MDP: Theoretical Framework for Decentralized Agents with Dynamic Dependenciescited by 5
  2. 2021Dimension-Free Rates for Natural Policy Gradient in Multi-Agent Reinforcement Learningcited by 4
  3. 2021Learning to Share in Multi-Agent Reinforcement Learningcited by 4
  4. 2022On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learningcited by 4
  5. 2023Major-Minor Mean Field Multi-Agent Reinforcement Learningcited by 4
  6. 2021Locality Matters: A Scalable Value Decomposition Approach for Cooperative Multi-Agent Reinforcement Learningcited by 3
  7. 2024Sample and Communication Efficient Fully Decentralized MARL Policy Evaluation via a New Approach: Local TD updatecited by 3
  8. 2024Approximate Global Convergence of Independent Learning in Multi-Agent Systemscited by 3
  9. 2024Scalable spectral representations for multi-agent reinforcement learning in network MDPscited by 3
  10. 2022Scalable and Sample Efficient Distributed Policy Gradient Algorithms in Multi-Agent Networked Systemscited by 2
  11. 2025Ego-centric Learning of Communicative World Models for Autonomous Drivingcited by 2
  12. 2025Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learningcited by 2
  13. 2021Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programmingcited by 1
  14. 2024Multi-agent Assessment with QoS Enhancement for HD Map Updates in a Vehicular Networkcited by 1
  15. 2025GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspectivecited by 1
  16. 2025Unifying Agent Interaction and World Information for Multi-agent Coordinationcited by 1
  17. 2025Transformer-Based Multi-Agent Reinforcement Learning for Networked Systems with Long-Range Interactionscited by 1
  18. 2026Decentralized Federated Learning With Energy Harvesting Devicescited by 1
  19. 2026Graphon Mean-Field Subsampling for Cooperative Heterogeneous Multi-Agent Reinforcement Learningcited by 1
  20. 2026A Unified Framework for Locality in Scalable MARLcited by 1
  21. 2026Revisiting Policy Gradients for Restricted Policy Classes: Escaping Myopic Local Optima with k-step Policy Gradientscited by 1
  22. 2026Scalable Constrained Multi-Agent Reinforcement Learning via State Augmentation and Consensus for Separable Dynamicscited by 1
  23. 2022Offsetting Unequal Competition through RL-assisted Incentive Schemescited by 0
  24. 2022Shaping Advice in Deep Reinforcement Learningcited by 0
  25. 2022General sum stochastic games with networked information flowscited by 0
  26. 2022Cooperative Actor-Critic via TD Error Aggregationcited by 0
  27. 2023An Algorithm For Adversary Aware Decentralized Networked MARLcited by 0
  28. 2024Distributed Policy Gradient for Linear Quadratic Networked Control with Limited Communication Rangecited by 0
  29. 2025LOD: Latent Objective Discovery in Heterogeneous Multi-Agent Reinforcement Learningcited by 0
  30. 2025Signal attenuation enables scalable decentralized multi-agent reinforcement learning over networkscited by 0
  31. 2026Explicit Credit Assignment through Local Rewards and Dependence Graphs in Multi-Agent Reinforcement Learningcited by 0
  32. 2026Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorizationcited by 0
  33. 2026Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedbackcited by 0
  34. 2026Aggregate in the Advantage, Not the Ratio: A Canonical-Form Analysis of Cooperative Multi-Agent Policy Optimizationcited by 0