# Distributed Detection of Adversarial Attacks in Multi-Agent Reinforcement Learning with Continuous Action Space

arXiv: https://arxiv.org/abs/2508.15764 · HTML: https://mlsanity.com/paper/2508.15764/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 21 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [Distributed Prioritized Experience Replay](https://mlsanity.com/paper/1803.00933/) (2018)
- [LSTM-based Encoder-Decoder for Multi-sensor Anomaly Detection](https://mlsanity.com/paper/1607.00148/) (2016)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Robust Deep Reinforcement Learning with Adversarial Attacks](https://mlsanity.com/paper/1712.03632/) (2017)
- [Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks](https://mlsanity.com/paper/1701.04143/) (2017)
- [Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing](https://mlsanity.com/paper/2102.07475/) (2021)
- [On the Robustness of Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2003.03722/) (2020)
- [Robust Multi-Agent Reinforcement Learning via Adversarial Regularization: Theoretical Foundation and Stable Algorithms](https://mlsanity.com/paper/2310.10810/) (2023)
- [Continuous Control with Action Quantization from Demonstrations](https://mlsanity.com/paper/2110.10149/) (2021)
- [Recurrent Off-policy Baselines for Memory-based Continuous Control](https://mlsanity.com/paper/2110.12628/) (2021)
- [Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2204.07932/) (2022)
- [Sequential Anomaly Detection using Inverse Reinforcement Learning](https://mlsanity.com/paper/2004.10398/) (2020)
- [Multi-Scale One-Class Recurrent Neural Networks for Discrete Event Sequence Anomaly Detection](https://mlsanity.com/paper/2008.13361/) (2020)
- [Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight](https://mlsanity.com/paper/1710.00814/) (2017)
- [Adversarial attacks in consensus-based multi-agent reinforcement learning](https://mlsanity.com/paper/2103.06967/) (2021)
- [Policy Entropy for Out-of-Distribution Classification](https://mlsanity.com/paper/2005.12069/) (2020)
- [Real-time Adversarial Perturbations against Deep Reinforcement Learning Policies: Attacks and Defenses](https://mlsanity.com/paper/2106.08746/) (2021)
- [A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning](https://mlsanity.com/paper/2109.09889/) (2021)
- [Robust Communicative Multi-Agent Reinforcement Learning with Active Defense](https://mlsanity.com/paper/2312.11545/) (2023)
- [Discretizing Continuous Action Space with Unimodal Probability Distributions for On-Policy Reinforcement Learning](https://mlsanity.com/paper/2408.00309/) (2024)
- [OIL-AD: An Anomaly Detection Framework for Sequential Decision Sequences](https://mlsanity.com/paper/2402.04567/) (2024)
- [Quickest Detection of Adversarial Attacks Against Correlated Equilibria](https://mlsanity.com/paper/2506.24040/) (2025)

## Abstract (arXiv)

We address the problem of detecting adversarial attacks against cooperative multi-agent reinforcement learning with continuous action space. We propose a decentralized detector that relies solely on the local observations of the agents and makes use of a statistical characterization of the normal behavior of observable agents. The proposed detector utilizes deep neural networks to approximate the normal behavior of agents as parametric multivariate Gaussian distributions. Based on the predicted density functions, we define a normality score and provide a characterization of its mean and variance. This characterization allows us to employ a two-sided CUSUM procedure for detecting deviations of the normality score from its mean, serving as a detector of anomalous behavior in real-time. We evaluate our scheme on various multi-agent PettingZoo benchmarks against different state-of-the-art attack methods, and our results demonstrate the effectiveness of our method in detecting impactful adversarial attacks. Particularly, it outperforms the discrete counterpart by achieving AUC-ROC scores of over 0.95 against the most impactful attacks in all evaluated environments.