mlsanityData to 7 Oct 2026

Data Poisoning to Fake a Nash Equilibrium in Markov Games

arXiv 2306.08041 ↗First posted 13 Jun 2023cs.MA cs.AI cs.CRcited by 1

On the map: LLM safety and security → Backdoor and data poisoning attacks

Abstract (arXiv)

We characterize offline data poisoning attacks on Multi-Agent Reinforcement Learning (MARL), where an attacker may change a data set in an attempt to install a (potentially fictitious) unique Markov-perfect Nash equilibrium for a two-player zero-sum Markov game. We propose the unique Nash set, namely the set of games, specified by their Q functions, with a specific joint policy being the unique Nash equilibrium. The unique Nash set is central to poisoning attacks because the attack is successful if and only if data poisoning pushes all plausible games inside the set. The unique Nash set generalizes the reward polytope commonly used in inverse reinforcement learning to MARL. For zero-sum Markov games, both the inverse Nash set and the set of plausible games induced by data are polytopes in the Q function space. We exhibit a linear program to efficiently compute the optimal poisoning attack. Our work sheds light on the structure of data poisoning attacks on offline MARL, a necessary step before one can design more robust MARL algorithms.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Backdoor and data poisoning attacks, one of 12 directions in LLM safety and security. Builds on 16 of its 29 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

16 of 29 references.

References · 16

  1. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  2. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
  3. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  4. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  5. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  6. 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
  7. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  8. 2019Policy Poisoning in Batch Reinforcement Learning and Controlcited by 70
  9. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  10. 2020Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamicscited by 38
  11. 2021Provably Efficient Black-Box Action Poisoning Attacks Against Reinforcement Learningcited by 29
  12. 2022When is Offline Two-Player Zero-Sum Markov Game Solvable?cited by 27
4 more
  1. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  2. 2020Policy Teaching in Reinforcement Learning via Environment Poisoning Attackscited by 24
  3. 2022Understanding the Limits of Poisoning Attacks in Episodic Reinforcement Learningcited by 21
  4. 2021Game Redesign in No-regret Game Playingcited by 8

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Minimally Modifying a Markov Game to Achieve Any Nash Equilibrium and Valuecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2022Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
  2. 2025Collapsing Sequence-Level Data-Policy Coverage via Poisoning Attack in Offline Reinforcement Learning
  3. 2025Optimal Perturbation Budget Allocation for Data Poisoning in Offline Reinforcement Learning
  4. 2023Offline Learning in Markov Games with General Function Approximation
  5. 2025Multi-level Certified Defense Against Poisoning Attacks in Offline Reinforcement Learning
  6. 2026Corruption-robust Offline Multi-agent Reinforcement Learning From Human Feedback
  7. 2025Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning
  8. 2024Offline Fictitious Self-Play for Competitive Games