mlsanityData to 7 Oct 2026

A Variational Approach to Mutual Information-Based Coordination for Multi-Agent Reinforcement Learning

arXiv 2303.00451 ↗First posted 1 Mar 2023cs.MA cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a new mutual information framework for multi-agent reinforcement learning to enable multiple agents to learn coordinated behaviors by regularizing the accumulated return with the simultaneous mutual information between multi-agent actions. By introducing a latent variable to induce nonzero mutual information between multi-agent actions and applying a variational bound, we derive a tractable lower bound on the considered MMI-regularized objective function. The derived tractable objective can be interpreted as maximum entropy reinforcement learning combined with uncertainty reduction of other agents actions. Applying policy iteration to maximize the derived lower bound, we propose a practical algorithm named variational maximum mutual information multi-agent actor-critic, which follows centralized learning with decentralized execution. We evaluated VM3-AC for several games requiring coordination, and numerical results show that VM3-AC outperforms other MARL algorithms in multi-agent tasks requiring high-quality coordination.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 21 of its 35 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

21 of 35 references.

References · 21

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  7. 2019The StarCraft Multi-Agent Challengecited by 589
  8. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
  9. 2019MAVEN: Multi-Agent Variational Explorationcited by 181
  10. 2019A Review of Cooperative Multi-Agent Deep Reinforcement Learningcited by 147
  11. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  12. 2020Off-Policy Multi-Agent Decomposed Policy Gradientscited by 97
9 more
  1. 2019Influence-Based Multi-Agent Explorationcited by 75
  2. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  3. 2019Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learningcited by 57
  4. 2020Value-Decomposition Multi-Agent Actor-Criticscited by 47
  5. 2022MASER: Multi-Agent Reinforcement Learning with Subgoals Generated from Experience Replay Buffercited by 34
  6. 2018Managing engineering systems with large state and action spaces through deep reinforcement learningcited by 32
  7. 2018Multi-Agent Common Knowledge Reinforcement Learningcited by 31
  8. 2019Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learningcited by 17
  9. 2020Multi-Agent Interactions Modeling with Correlated Policiescited by 13

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 3.

5 papers.

  1. 2023Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learningcited by 13
  2. 2025Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learningcited by 10
  3. 2023LESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Frameworkcited by 7
  4. 2025B3C: A Minimalist Approach to Offline Multi-Agent Reinforcement Learningcited by 1
  5. 2025When Assisting One Disempowers Anothercited by 0