mlsanityData to 7 Oct 2026

ROMAX: Certifiably Robust Deep Multiagent Reinforcement Learning via Convex Relaxation

arXiv 2109.06795 ↗First posted 14 Sep 2021cs.LGcited by 17

Not on the map. Why?

Abstract (arXiv)

In a multirobot system, a number of cyber-physical attacks (e.g., communication hijack, observation perturbations) can challenge the robustness of agents. This robustness issue worsens in multiagent reinforcement learning because there exists the non-stationarity of the environment caused by simultaneously learning agents whose changing policies affect the transition and reward functions. In this paper, we propose a minimax MARL approach to infer the worst-case policy update of other agents. As the minimax formulation is computationally intractable to solve, we apply the convex relaxation of neural networks to solve the inner minimization problem. Such convex relaxation enables robustness in interacting with peer agents that may have significantly different behaviors and also achieves a certified bound of the original optimization problem. We evaluate our approach on multiple mixed cooperative-competitive tasks and show that our method outperforms the previous state of the art approaches on this topic.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 30 of its 46 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

30 of 46 references.

References · 30

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  3. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  4. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  7. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  8. 2017Population Based Training of Neural Networkscited by 351
  9. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  10. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  11. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  12. 2017Learning with Opponent-Learning Awarenesscited by 218
18 more
  1. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  2. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  3. 2018Mean Field Multi-Agent Reinforcement Learningcited by 202
  4. 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 178
  5. 2018Efficient Formal Safety Analysis of Neural Networkscited by 165
  6. 2019A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networkscited by 124
  7. 2017Emergent Complexity via Multi-Agent Competitioncited by 117
  8. 2017Continuous Adaptation via Meta-Learning in Nonstationary and Competitive Environmentscited by 106
  9. 2017A Survey of Learning in Multiagent Environments: Dealing with Non-Stationaritycited by 93
  10. 2020The Complexity of Constrained Min-Max Optimizationcited by 81
  11. 2019Probabilistic Recursive Reasoning for Multi-Agent Reinforcement Learningcited by 68
  12. 2018Stable Opponent Shaping in Differentiable Gamescited by 49
  13. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  14. 2018DiCE: The Infinitely Differentiable Monte-Carlo Estimatorcited by 46
  15. 2020A Policy Gradient Algorithm for Learning to Learn in Multiagent Reinforcement Learningcited by 40
  16. 2016Learning Nash Equilibrium for General-Sum Markov Games from Batch Datacited by 20
  17. 2018Balancing Two-Player Stochastic Games with Soft Q-Learningcited by 5
  18. 2020Decentralized Adaptive Control for Collaborative Manipulation of Rigid Bodiescited by 3

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 7; 2024: 3; 2025: 3.

17 papers.

Sort:

  1. 2023Robust Multi-Agent Reinforcement Learning with State Uncertaintycited by 35
  2. 2024Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planescited by 33
  3. 2022What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?cited by 21
  4. 2022RoMFAC: A robust mean-field actor-critic reinforcement learning against adversarial perturbations on statescited by 16
  5. 2025Clip-and-Verify: Linear Constraint-Driven Domain Clipping for Accelerating Neural Network Verificationcited by 9
  6. 2022Sparse Adversarial Attack in Multi-agent Reinforcement Learningcited by 8
  7. 2023Robust Multi-Agent Reinforcement Learning by Mutual Information Regularizationcited by 6
  8. 2022Game-Theoretical Perspectives on Active Equilibria: A Preferred Solution Concept over Nash Equilibriacited by 5
  9. 2023Multi-Agent Reinforcement Learning: Methods, Applications, Visionary Prospects, and Challengescited by 5
  10. 2023Robust Multi-agent Communication via Multi-view Message Certificationcited by 3
  11. 2023Wasserstein Distributionally Robust Control Barrier Function using Conditional Value-at-Risk with Differentiable Convex Programmingcited by 2
  12. 2023Distributionally Safe Reinforcement Learning under Model Uncertainty: A Single-Level Approach by Differentiable Convex Programmingcited by 1
5 more
  1. 2023Robustness to Multi-Modal Environment Uncertainty in MARL using Curriculum Learningcited by 1
  2. 2024Towards Fault Tolerance in Multi-Agent Reinforcement Learningcited by 1
  3. 2025Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learningcited by 1
  4. 2024Noisy Zero-Shot Coordination: Breaking The Common Knowledge Assumption In Zero-Shot Coordination Gamescited by 0
  5. 2025SAJA: A State-Action Joint Attack Framework on Multi-Agent Deep Reinforcement Learningcited by 0