mlsanityData to 14 Oct 2026

Vulnerable Agent Identification in Large-Scale Multi-Agent Reinforcement Learning

arXiv 2509.15103 ↗First posted 18 Sep 2025cs.MA cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Partial agent failure becomes inevitable when systems scale up, making it crucial to identify the subset of agents whose failure causes worst-case system performance degradations. We study this Vulnerable Agent Identification (VAI) problem in large-scale multi-agent reinforcement learning (MARL). We frame VAI as a Hierarchical Adversarial Decentralized Mean Field Control (HAD-MFC), where the upper level selects vulnerable agents as an NP-hard task and the lower level learns their worst-case adversarial policies via mean-field MARL. The two problems are coupled together, making HAD-MFC difficult to solve. To handle this, we first decouple the hierarchical process by Fenchel-Rockafellar transform, resulting a regularized mean-field Bellman operator for upper level that enables independent learning at each level, thus reducing computational complexity. We next reformulate the upper-level NP-hard problem as an MDP with dense rewards, allowing sequential identification of vulnerable agents via greedy and RL algorithms. This decomposition provably preserves the optimal solution. Experiments show our method effectively identifies more vulnerable agents in large-scale MARL and the rule-based system, fooling system into worse failures, and reveals the vulnerability of each agent in large systems. Code available at https://github.com/Waken-dream/VAI

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 71 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

36 of 71 references.

References · 36

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  3. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  4. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  5. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  6. 2015Deep Reinforcement Learning in Large Discrete Action Spacescited by 161
  7. 2020Reinforcement Learning via Fenchel-Rockafellar Dualitycited by 100
  8. 2017MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligencecited by 71
  9. 2019Model-Free Mean-Field Reinforcement Learning: Mean-Field MDP and Mean-Field Q-Learningcited by 59
  10. 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 54
  11. 2020Mean-Field Controls with Q-learning for Cooperative MARL: Convergence and Complexity Analysiscited by 49
  12. 2020Unified Reinforcement Q-Learning for Mean Field Game and Control Problemscited by 49
24 more
  1. 2018Deep Reinforcement Learning for Swarm Systemscited by 45
  2. 2020On the Robustness of Cooperative Multi-Agent Reinforcement Learningcited by 39
  3. 2021Efficient Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 38
  4. 2022Scalable Deep Reinforcement Learning Algorithms for Mean Field Gamescited by 38
  5. 2021Scaling up Mean Field Games with Online Mirror Descentcited by 32
  6. 2021On the Approximation of Cooperative Heterogeneous Multi-Agent Reinforcement Learning (MARL) using Mean Field Control (MFC)cited by 32
  7. 2020Controlling Graph Dynamics with Reinforcement Learning and Graph Neural Networkscited by 30
  8. 2023Deep Graph Representation Learning and Optimization for Influence Maximizationcited by 27
  9. 2020Multi Type Mean Field Reinforcement Learningcited by 26
  10. 2024Sample-Efficient Robust Multi-Agent Reinforcement Learning in the Face of Environmental Uncertaintycited by 25
  11. 2022Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learningcited by 21
  12. 2023Byzantine Robust Cooperative Multi-Agent Reinforcement Learning as a Bayesian Gamecited by 18
  13. 2023Attacking Cooperative Multi-Agent Reinforcement Learning by Adversarial Minority Influencecited by 17
  14. 2020Partially Observable Mean Field Reinforcement Learningcited by 16
  15. 2022A Survey on Influence Maximization: From an ML-Based Combinatorial Optimizationcited by 15
  16. 2022Efficient Model-based Multi-agent Reinforcement Learning via Optimistic Equilibrium Computationcited by 14
  17. 2021Decentralized Mean Field Gamescited by 12
  18. 2018A Survey on Influence Maximization in a Social Networkcited by 11
  19. 2022ToupleGDD: A Fine-Designed Solution of Influence Maximization by Deep Reinforcement Learningcited by 11
  20. 2023Robustness Testing for Multi-Agent Reinforcement Learning: State Perturbations on Critical Agentscited by 7
  21. 2023Learning Decentralized Partially Observable Mean Field Control for Artificial Collective Behaviorcited by 7
  22. 2022Learning Correlated Equilibria in Mean-Field Gamescited by 6
  23. 2023Major-Minor Mean Field Multi-Agent Reinforcement Learningcited by 4
  24. 2021Online Markov Decision Processes with Non-oblivious Strategic Adversarycited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning
  2. 2025Bi-level Mean Field: Dynamic Grouping for Large-Scale MARL
  3. 2025Robust and Diverse Multi-Agent Learning via Rational Policy Gradient
  4. 2025Constrained Black-Box Attacks Against Cooperative Multi-Agent Reinforcement Learning
  5. 2025Policy Disruption in Reinforcement Learning:Adversarial Attack with Large Language Models and Critical State Identification
  6. 2024CuDA2: An approach for Incorporating Traitor Agents into Cooperative Multi-Agent Systems
  7. 2024Mean-Field Sampling for Cooperative Multi-Agent Reinforcement Learning
  8. 2021Adversarial Inverse Reinforcement Learning for Mean Field Games