mlsanityData to 14 Oct 2026

Reducing Variance Caused by Communication in Decentralized Multi-agent Deep Reinforcement Learning

arXiv 2502.06261 ↗First posted 10 Feb 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

In decentralized multi-agent deep reinforcement learning (MADRL), communication can help agents to gain a better understanding of the environment to better coordinate their behaviors. Nevertheless, communication may involve uncertainty, which potentially introduces variance to the learning of decentralized agents. In this paper, we focus on a specific decentralized MADRL setting with communication and conduct a theoretical analysis to study the variance that is caused by communication in policy gradients. We propose modular techniques to reduce the variance in policy gradients during training. We adopt our modular techniques into two existing algorithms for decentralized MADRL with communication and evaluate them on multiple tasks in the StarCraft Multi-Agent Challenge and Traffic Junction domains. The results show that decentralized MADRL communication methods extended with our proposed techniques not only achieve high-performing agents but also reduce variance in policy gradients during training.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 10 of its 12 references that are in the map. First posted on arXiv 10 Feb 2025.

Builds on

10 of 12 references.

References · 10

  1. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  2. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  3. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 175
  4. 2022A Survey of Multi-Agent Deep Reinforcement Learning with Communicationcited by 104
  5. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 85
  6. 2019Multi-Agent Game Abstraction via Graph Attention Neural Networkcited by 82
  7. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 68
  8. 2021Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learningcited by 56
  9. 2021Settling the Variance of Multi-Agent Policy Gradientscited by 45
  10. 2019Learning to Communicate in Multi-Agent Reinforcement Learning : A Reviewcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Dynamic Generation of Multi-LLM Agents Communication Topologies with Graph Diffusion Modelscited by 11

Similar, not cited

Close in wording, no citation either way

  1. 2025Fully-Decentralized MADDPG with Networked Agents
  2. 2026Communication Methods in Multi-Agent Reinforcement Learning
  3. 2024Joint Intrinsic Motivation for Coordinated Exploration in Multi-Agent Deep Reinforcement Learning
  4. 2025Deep Reinforcement Learning for Multi-Agent Coordination
  5. 2025Learning what to say and how precisely: Efficient Communication via Differentiable Discrete Communication Learning
  6. 2025A Communication-Efficient Decentralized Actor-Critic Algorithm
  7. 2024Fully Independent Communication in Multi-Agent Reinforcement Learning
  8. 2024ClusterComm: Discrete Communication in Decentralized MARL using Internal Representation Clustering