mlsanityData to 7 Oct 2026

Distributionally Robust Cooperative Multi-Agent Reinforcement Learning via Robust Value Factorization

arXiv 2602.11437 ↗First posted 11 Feb 2026cs.AI cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

Cooperative multi-agent reinforcement learning (MARL) commonly adopts centralized training with decentralized execution, where value-factorization methods enforce the individual-global-maximum (IGM) principle so that decentralized greedy actions recover the team-optimal joint action. However, the reliability of this recipe in real-world settings remains unreliable due to environmental uncertainties arising from the sim-to-real gap, model mismatch, and system noise. We address this gap by introducing Distributionally robust IGM (DrIGM), a principle that requires each agent's robust greedy action to align with the robust team-optimal joint action. We show that DrIGM holds for a novel definition of robust individual action values, which is compatible with decentralized greedy execution and yields a provable robustness guarantee for the whole system. Building on this foundation, we derive DrIGM-compliant robust variants of existing value-factorization architectures (e.g., VDN/QMIX/QTRAN) that (i) train on robust Q-targets, (ii) preserve scalability, and (iii) integrate seamlessly with existing codebases without bespoke per-agent reward shaping. Empirically, on high-fidelity SustainGym simulators and a StarCraft game environment, our methods consistently improve out-of-distribution performance. Code and data are available at https://github.com/crqu/robust-coMARL.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 81 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

49 of 81 references.

References · 49

  1. 2019The StarCraft Multi-Agent Challengecited by 589
  2. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  3. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  4. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 475
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 457
  6. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  7. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 372
  8. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  9. 2015Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulationscited by 285
  10. 2017StarCraft II: A New Challenge for Reinforcement Learningcited by 268
  11. 2020QPLEX: Duplex Dueling Multi-Agent Q-Learningcited by 267
  12. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 206
37 more
  1. 2023SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Modelscited by 151
  2. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  3. 2022Robust Reinforcement Learning using Offline Datacited by 95
  4. 2013Data-Driven Robust Optimizationcited by 84
  5. 2020Qatten: A General Framework for Cooperative Multiagent Reinforcement Learningcited by 81
  6. 2019Robust Reinforcement Learning for Continuous Control with Model Misspecificationcited by 76
  7. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
  8. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  9. 2017Reinforcement Learning under Model Mismatchcited by 52
  10. 2023Double Pessimism is Provably Efficient for Distributionally Robust Offline Reinforcement Learning: Generic Algorithm and Robust Partial Coveragecited by 51
  11. 2023The Curious Price of Distributional Robustness in Reinforcement Learning with a Generative Modelcited by 50
  12. 2020Partial Policy Iteration for L1-Robust Markov Decision Processescited by 49
  13. 2020Finite-Sample Guarantees for Wasserstein Distributionally Robust Optimization: Breaking the Curse of Dimensionalitycited by 49
  14. 2020Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Rewardcited by 46
  15. 2023Improved Sample Complexity Bounds for Distributionally Robust Reinforcement Learningcited by 44
  16. 2019Distributionally Robust Reinforcement Learningcited by 40
  17. 2020Distributional Robustness and Regularization in Reinforcement Learningcited by 39
  18. 2020Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learningcited by 38
  19. 2018Soft-Robust Actor-Critic Policy-Gradientcited by 37
  20. 2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 35
  21. 2023Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithmscited by 25
  22. 2024Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximationcited by 25
  23. 2024Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithmscited by 24
  24. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 24
  25. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 21
  26. 2022Online Policy Optimization for Robust MDPcited by 19
  27. 2023RiskQ: Risk-sensitive Multi-Agent Reinforcement Learning Value Factorizationcited by 19
  28. 2023Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Gamecited by 18
  29. 2020Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networkscited by 16
  30. 2024Distributionally Robust Constrained Reinforcement Learning under Strong Dualitycited by 15
  31. 2019DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learningcited by 13
  32. 2023Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agentscited by 7
  33. 2023Robust Communicative Multi-Agent Reinforcement Learning with Active Defensecited by 4
  34. 2023Enhancing the Robustness of QMIX against State-adversarial Attackscited by 3
  35. 2025Distributionally Robust Multi-Agent Reinforcement Learning for Dynamic Chute Mappingcited by 2
  36. 2025Decision-Dependent Distributionally Robust Optimization with Application to Dynamic Pricingcited by 2
  37. 2023Decision-dependent Distributionally Robust Optimizationcited by 1

Built on by

Not cited yet.