mlsanityData to 14 Oct 2026

Adversarial Policies Beat Superhuman Go AIs

arXiv 2211.00241 ↗First posted 1 Nov 2022cs.LG cs.AI cs.CRcited by 29

Not on the map. Why?

Abstract (arXiv)

We attack the state-of-the-art Go-playing AI system KataGo by training adversarial policies against it, achieving a >97% win rate against KataGo running at superhuman settings. Our adversaries do not win by playing Go well. Instead, they trick KataGo into making serious blunders. Our attack transfers zero-shot to other superhuman Go-playing AIs, and is comprehensible to the extent that human experts can implement it without algorithmic assistance to consistently beat superhuman AIs. The core vulnerability uncovered by our attack persists even in KataGo agents adversarially trained to defend against our attack. Our results demonstrate that even superhuman AI systems may harbor surprising failure modes. Example games are available https://goattack.far.ai/.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 20 of its 86 references that are in the map. First posted on arXiv 1 Nov 2022.

Builds on

20 of 86 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  6. 2019On Evaluating Adversarial Robustnesscited by 447
  7. 2020Meta Pseudo Labelscited by 374
  8. 2017Adversarial Attacks on Neural Network Policiescited by 320
  9. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  10. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  11. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  12. 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
8 more
  1. 2020Real World Games Look Like Spinning Topscited by 66
  2. 2019Accelerating Self-Play Learning in Gocited by 64
  3. 2020Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learningcited by 61
  4. 2020From Poincar'e Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularizationcited by 61
  5. 2019ELF OpenGo: An Analysis and Open Reimplementation of AlphaZerocited by 39
  6. 2018CIFAR10 to Compare Visual Recognition Performance between Deep Neural Networks and Humanscited by 14
  7. 2022Reducing Exploitability with Population Based Trainingcited by 4
  8. 2022Imitating Opponent to Win: Adversarial Policy Imitation Learning in Two-player Competitive Gamescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 7; 2025: 12; 2026: 4.

29 papers.

Sort:

By area:

  1. 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 178
  2. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  3. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  4. 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 59
  5. 2025Virtual Agent Economiescited by 21
  6. 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systemscited by 7
  7. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6
  8. 2025Foundation Model Self-Play: Open-Ended Strategy Innovation via Foundation Modelscited by 6
  9. 2025UNIDOOR: A Universal Framework for Action-Level Backdoor Attacks in Deep Reinforcement Learningcited by 5
  10. 2024Multi-Agent Diagnostics for Robustness via Illuminated Diversitycited by 4
  11. 2025TrojanTO: Action-Level Backdoor Attacks against Trajectory Optimization Modelscited by 3
  12. 2023Minimax Exploiter: A Data Efficient Approach for Competitive Self-Playcited by 2
17 more
  1. 2025The Structural Safety Generalization Problemcited by 2
  2. 2025Look-ahead Reasoning with a Learned Model in Imperfect Information Gamescited by 2
  3. 2023On existence, uniqueness and scalability of adversarial robustness measures for AI classifierscited by 1
  4. 2023Responsible Emergent Multi-Agent Behaviorcited by 1
  5. 2024Demystifying MuZero Planning: Interpreting the Learned Modelcited by 1
  6. 2025Beyond Accuracy: A Geometric Stability Analysis of Large Language Models in Chess Evaluationcited by 1
  7. 2026Angel or Demon: Investigating the Plasticity Interventions' Impact on Backdoor Threats in Deep Reinforcement Learningcited by 1
  8. 2024Games of Knightian Uncertainty as AGI testbedscited by 0
  9. 2024Bridging Local and Global Knowledge via Transformer in Board Gamescited by 0
  10. 2025Search-contempt: a hybrid MCTS algorithm for training AlphaZero-like engines with better computational efficiencycited by 0
  11. 2025LLM world models are mental: Output layer evidence of brittle world model use in LLM mechanical reasoningcited by 0
  12. 2025Relevance-Zone Reduction in Game Solvingcited by 0
  13. 2025Outbidding and Outbluffing Elite Humans: Mastering Liar's Poker via Self-Play and Reinforcement Learningcited by 0
  14. 2025A Study of Solving Life-and-Death Problems in Go Using Relevance-Zone Based Solverscited by 0
  15. 2026Nemobot Games: Crafting Strategic AI Gaming Agents for Interactive Learning with Large Language Modelscited by 0
  16. 2026Synthetic Counteradaptation: A Principle of Human-AI Co-evolutioncited by 0
  17. 2026Unsound Search with Policy and Value Networks in Legends of Code and Magiccited by 0