mlsanityData to 14 Oct 2026

Robust Multi-Agent Reinforcement Learning by Mutual Information Regularization

arXiv 2310.09833 ↗First posted 15 Oct 2023cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

In multi-agent reinforcement learning (MARL), ensuring robustness against unpredictable or worst-case actions by allies is crucial for real-world deployment. Existing robust MARL methods either approximate or enumerate all possible threat scenarios against worst-case adversaries, leading to computational intensity and reduced robustness. In contrast, human learning efficiently acquires robust behaviors in daily life without preparing for every possible threat. Inspired by this, we frame robust MARL as an inference problem, with worst-case robustness implicitly optimized under all threat scenarios via off-policy evaluation. Within this framework, we demonstrate that Mutual Information Regularization as Robust Regularization (MIR3) during routine training is guaranteed to maximize a lower bound on robustness, without the need for adversaries. Further insights show that MIR3 acts as an information bottleneck, preventing agents from over-reacting to others and aligning policies with robust action priors. In the presence of worst-case adversaries, our MIR3 significantly surpasses baseline methods in robustness and training efficiency while maintaining cooperative performance in StarCraft II and robot swarm control. When deploying the robot swarm control algorithm in the real world, our method also outperforms the best baseline by 14.29%.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 44 of its 79 references that are in the map. First posted on arXiv 15 Oct 2023.

Builds on

44 of 79 references.

References · 44

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  7. 2016Deep Variational Information Bottleneckcited by 930
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  9. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  10. 2018MINE: Mutual Information Neural Estimationcited by 683
  11. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  12. 2019The StarCraft Multi-Agent Challengecited by 597
32 more
  1. 2019On Variational Bounds of Mutual Informationcited by 559
  2. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
  3. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  4. 2019Information-Theoretic Considerations in Batch Reinforcement Learningcited by 287
  5. 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 235
  6. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  7. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  8. 2021Trust Region Policy Optimisation in Multi-Agent Reinforcement Learningcited by 186
  9. 2019MAVEN: Multi-Agent Variational Explorationcited by 185
  10. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176
  11. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  12. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 119
  13. 2019Influence-Based Multi-Agent Explorationcited by 75
  14. 2020Batch Policy Learning in Average Reward Markov Decision Processescited by 69
  15. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  16. 2021Exploration in Deep Reinforcement Learning: From Single-Agent to Multiagent Domaincited by 58
  17. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 47
  18. 2018Deep Reinforcement Learning for Swarm Systemscited by 45
  19. 2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 36
  20. 2022PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaborationcited by 23
  21. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 22
  22. 2021Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learningcited by 19
  23. 2021Mis-spoke or mis-lead: Achieving Robustness in Multi-Agent Communicative Reinforcement Learningcited by 18
  24. 2023Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Gamecited by 18
  25. 2021ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxationcited by 17
  26. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 17
  27. 2023Robust multi-agent coordination via evolutionary generation of auxiliary adversarial attackerscited by 15
  28. 2022MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Treescited by 11
  29. 2022A Robust and Constrained Multi-Agent Reinforcement Learning Electric Vehicle Rebalancing Method in AMoD Systemscited by 11
  30. 2019Signal Instructed Coordination in Cooperative Multi-agent Reinforcement Learningcited by 4
  31. 2021Online Markov Decision Processes with Non-oblivious Strategic Adversarycited by 3
  32. 2021Influence-Based Reinforcement Learning for Intrinsically-Motivated Agentscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 2.

6 papers.

  1. 2024Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Taskscited by 13
  2. 2025Diffusion Secant Alignment for Score-Based Density Ratio Estimationcited by 3
  3. 2025Robust and Efficient Communication in Multi-Agent Reinforcement Learningcited by 2
  4. 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learningcited by 1
  5. 2026Can LLM Agents Sustain Long-Horizon Organizational Dynamics?cited by 1
  6. 2025Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Directioncited by 0