mlsanityData to 7 Oct 2026

Certified Policy Smoothing for Cooperative Multi-Agent Reinforcement Learning

arXiv 2212.11746 ↗First posted 22 Dec 2022cs.LG cs.MAcited by 7

Not on the map. Why?

Abstract (arXiv)

Cooperative multi-agent reinforcement learning (c-MARL) is widely applied in safety-critical scenarios, thus the analysis of robustness for c-MARL models is profoundly important. However, robustness certification for c-MARLs has not yet been explored in the community. In this paper, we propose a novel certification method, which is the first work to leverage a scalable approach for c-MARLs to determine actions with guaranteed certified bounds. c-MARL certification poses two key challenges compared with single-agent systems: (i) the accumulated uncertainty as the number of agents increases; (ii) the potential lack of impact when changing the action of a single agent into a global team reward. These challenges prevent us from directly using existing algorithms. Hence, we employ the false discovery rate (FDR) controlling procedure considering the importance of each agent to certify per-state robustness and propose a tree-search-based algorithm to find a lower bound of the global reward under the minimal certified perturbation. As our method is general, it can also be applied in single-agent environments. We empirically show that our certification bounds are much tighter than state-of-the-art RL certification solutions. We also run experiments on two popular c-MARL algorithms: QMIX and VDN, in two different environments, with two and four agents. The experimental results show that our method produces meaningful guaranteed robustness for all models and environments. Our tool CertifyCMARL is available at https://github.com/TrustAI/CertifyCMA

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 22 Dec 2022.

Builds on

32 of 50 references.

References · 32

  1. 2013Intriguing properties of neural networkscited by 5,114
  2. 2016Adversarial examples in the physical worldcited by 1,762
  3. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  4. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  5. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  6. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 490
  7. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  8. 2017Adversarial Attacks on Neural Network Policiescited by 314
  9. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  10. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  11. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  12. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
20 more
  1. 2017Delving into adversarial attacks on deep policiescited by 104
  2. 2018Reachability Analysis of Deep Neural Networks with Provable Guaranteescited by 93
  3. 2017Vulnerability of Deep Reinforcement Learning to Policy Induction Attackscited by 91
  4. 2014Optimal Inference After Model Selectioncited by 63
  5. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
  6. 2022Enhancing Adversarial Training with Second-Order Statistics of Weightscited by 42
  7. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  8. 2020Enforcing robust control guarantees within neural network policiescited by 40
  9. 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
  10. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
  11. 2018A Game-Based Approximate Verification of Deep Neural Networks with Provable Guaranteescited by 33
  12. 2019Optimal Attacks on Reinforcement Learning Policiescited by 21
  13. 2022PRoA: A Probabilistic Robustness Assessment against Functional Perturbationscited by 19
  14. 2020Generalizing Universal Adversarial Attacks Beyond Additive Perturbationscited by 17
  15. 2020Extensions and limitations of randomized smoothing for robustness guaranteescited by 15
  16. 2022DIMBA: Discretely Masked Black-Box Attack in Single Object Trackingcited by 11
  17. 2022Certifiably Robust Policy Learning against Adversarial Communication in Multi-agent Systemscited by 7
  18. 20223DVerifier: Efficient Robustness Verification for 3D Point Cloud Modelscited by 5
  19. 2022Understanding Adversarial Robustness of Vision Transformers via Cauchy Problemcited by 4
  20. 2018Cooperative Multi-Agent Reinforcement Learning for Low-Level Wireless Communicationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 3; 2026: 1.

7 papers.

  1. 2023Randomized Adversarial Training via Taylor Expansioncited by 28
  2. 2024Boosting Adversarial Training via Fisher-Rao Norm-based Regularizationcited by 7
  3. 2025Position: Certified Robustness Does Not (Yet) Imply Model Securitycited by 4
  4. 2025Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifiercited by 2
  5. 2023Model-Agnostic Reachability Analysis on Deep Neural Networkscited by 1
  6. 2025Principal Eigenvalue Regularization for Improved Worst-Class Certified Robustness of Smoothed Classifierscited by 0
  7. 2026Interpretable but Fragile? Robustness of Concept Bottlenecks under Geometric-Semantic Perturbationscited by 0