mlsanityData to 7 Oct 2026

Implicit Poisoning Attacks in Two-Agent Reinforcement Learning: Adversarial Policies for Training-Time Attacks

arXiv 2302.13851 ↗First posted 27 Feb 2023cs.LG cs.AI cs.CRcited by 14

Not on the map. Why?

Abstract (arXiv)

In targeted poisoning attacks, an attacker manipulates an agent-environment interaction to force the agent into adopting a policy of interest, called target policy. Prior work has primarily focused on attacks that modify standard MDP primitives, such as rewards or transitions. In this paper, we study targeted poisoning attacks in a two-agent setting where an attacker implicitly poisons the effective environment of one of the agents by modifying the policy of its peer. We develop an optimization framework for designing optimal attacks, where the cost of the attack measures how much the solution deviates from the assumed default policy of the peer agent. We further study the computational properties of this optimization framework. Focusing on a tabular setting, we show that in contrast to poisoning attacks based on MDP primitives (transitions and (unbounded) rewards), which are always feasible, it is NP-hard to determine the feasibility of implicit poisoning attacks. We provide characterization results that establish sufficient conditions for the feasibility of the attack problem, as well as an upper and a lower bound on the optimal cost of the attack. We propose two algorithmic approaches for finding an optimal adversarial policy: a model-based approach with tabular policies and a model-free approach with parametric/neural policies. We showcase the efficacy of the proposed algorithms through experiments.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 27 Feb 2023.

Builds on

45 of 64 references.

References · 45

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2013Intriguing properties of neural networkscited by 5,114
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  6. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  7. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  8. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  9. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  10. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  11. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  12. 2017Adversarial Attacks on Neural Network Policiescited by 314
33 more
  1. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  2. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 235
  3. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  4. 2020PettingZoo: Gym for Multi-Agent Reinforcement Learningcited by 209
  5. 2017A Rewriting System for Convex Optimization Problemscited by 178
  6. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 163
  7. 2017Robust Deep Reinforcement Learning with Adversarial Attackscited by 152
  8. 2017Neural Trojanscited by 151
  9. 2021Robust Reinforcement Learning on State Observations with Learned Optimal Adversarycited by 133
  10. 2017Delving into adversarial attacks on deep policiescited by 104
  11. 2018Is feature selection secure against training data poisoning?cited by 88
  12. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  13. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  14. 2016Data Poisoning Attacks on Factorization-Based Collaborative Filteringcited by 78
  15. 2019Corruption-robust exploration in episodic reinforcement learningcited by 72
  16. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 72
  17. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  18. 2021BACKDOORL: Backdoor Attack against Competitive Reinforcement Learningcited by 66
  19. 2020Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learningcited by 62
  20. 2021CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothingcited by 41
  21. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
  22. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  23. 2017Whatever Does Not Kill Deep Reinforcement Learning, Makes It Strongercited by 35
  24. 2020Stop-and-Go: Exploring Backdoor Attacks on Deep Reinforcement Learning-based Traffic Congestion Control Systemscited by 35
  25. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  26. 2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
  27. 2022COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attackscited by 25
  28. 2020Policy Teaching in Reinforcement Learning via Environment Poisoning Attackscited by 24
  29. 2022Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learningcited by 21
  30. 2019Design of intentional backdoors in sequential modelscited by 20
  31. 2021Defense Against Reward Poisoning Attacks in Reinforcement Learningcited by 20
  32. 2022Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learningcited by 19
  33. 2022Admissible Policy Teaching through Reward Designcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 5; 2026: 4.

14 papers.

Sort:

By area:

  1. 2024Performative Reinforcement Learning in Gradually Shifting Environmentscited by 13
  2. 2023Efficient Adversarial Attacks on Online Multi-agent Reinforcement Learningcited by 9
  3. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
  4. 2025Fox in the Henhouse: Supply-Chain Backdoor Attacks Against Reinforcement Learningcited by 5
  5. 2024Corruption-Robust Offline Two-Player Zero-Sum Markov Gamescited by 4
  6. 2023Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learningcited by 2
  7. 2025Policy Teaching via Data Poisoning in Learning from Human Preferencescited by 2
  8. 2026MaMa: A Game-Theoretic Approach for Designing Safe Agentic Systemscited by 2
  9. 2023Optimal Cost Constrained Adversarial Attacks For Multiple Agent Systemscited by 1
  10. 2025Bilevel Optimization for Covert Memory Tampering in Heterogeneous Multi-Agent Architectures (XAMT)cited by 1
  11. 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learningcited by 1
  12. 2025Robust In-Context Reinforcement Learning Under Reward Poisoning Attackscited by 0
2 more
  1. 2026When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPscited by 0
  2. 2026Local and Global Stability in Performative Reinforcement Learningcited by 0