mlsanityData to 14 Oct 2026

Are AlphaZero-like Agents Robust to Adversarial Perturbations?

arXiv 2211.03769 ↗First posted 7 Nov 2022cs.AI cs.LG cs.ROcited by 14

Not on the map. Why?

Abstract (arXiv)

The success of AlphaZero (AZ) has demonstrated that neural-network-based Go AIs can surpass human performance by a large margin. Given that the state space of Go is extremely large and a human player can play the game from any legal state, we ask whether adversarial states exist for Go AIs that may lead them to play surprisingly wrong actions. In this paper, we first extend the concept of adversarial examples to the game of Go: we generate perturbed states that are ``semantically'' equivalent to the original state by adding meaningless moves to the game, and an adversarial state is a perturbed state leading to an undoubtedly inferior action that is obvious even for Go beginners. However, searching the adversarial state is challenging due to the large, discrete, and non-differentiable search space. To tackle this challenge, we develop the first adversarial attack on Go AIs that can efficiently search for adversarial states by strategically reducing the search space. This method can also be extended to other board games such as NoGo. Experimentally, we show that the actions taken by both Policy-Value neural network (PV-NN) and Monte Carlo tree search (MCTS) can be misled by adding one or two meaningless stones; for example, on 58% of the AlphaGo Zero self-play games, our method can make the widely used KataGo agent with 50 simulations of MCTS plays a losing action by adding two meaningless stones. We additionally evaluated the adversarial examples found by our algorithm with amateur human Go players and 90% of examples indeed lead the Go agent to play an obviously inferior action. Our code is available at urlhttps://PaperCode.cc/GoAttack.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 28 of its 40 references that are in the map. First posted on arXiv 7 Nov 2022.

Builds on

28 of 40 references.

References · 28

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2013Intriguing properties of neural networkscited by 5,155
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  6. 2019Natural Adversarial Examplescited by 1,408
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  8. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  9. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 491
  10. 2018Generating Natural Language Adversarial Examplescited by 424
  11. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  12. 2017Adversarial Attacks on Neural Network Policiescited by 320
16 more
  1. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  2. 2020Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observationscited by 238
  3. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  4. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  5. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  6. 2017EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examplescited by 155
  7. 2017Delving into adversarial attacks on deep policiescited by 104
  8. 2019Accelerating Self-Play Learning in Gocited by 64
  9. 2019ELF OpenGo: An Analysis and Open Reimplementation of AlphaZerocited by 39
  10. 2018Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Datacited by 38
  11. 2018Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classificationcited by 35
  12. 2019Characterizing Attacks on Deep Reinforcement Learningcited by 31
  13. 2020Policy Teaching in Reinforcement Learning via Environment Poisoning Attackscited by 24
  14. 2020Accelerating and Improving AlphaZero Using Population Based Trainingcited by 12
  15. 2021A Novel Approach to Solving Goal-Achieving Problems for Board Gamescited by 7
  16. 2020Learning to Stop: Dynamic Simulation Monte-Carlo Tree Searchcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 7; 2024: 4; 2025: 3.

14 papers.

Sort:

By area:

  1. 2023Evaluating Superhuman Models with Consistency Checkscited by 47
  2. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  3. 2024Consistency Checks for Language Model Forecasterscited by 27
  4. 2023Can Agents Run Relay Race with Strangers? Generalization of RL to Out-of-Distribution Trajectoriescited by 9
  5. 2023Multi-scale Diffusion Denoised Smoothingcited by 9
  6. 2023Accelerating Monte Carlo Tree Search with Probability Tree State Abstractioncited by 3
  7. 2023Game Solving with Online Fine-Tuningcited by 3
  8. 2024Adversarial Robustification via Text-to-Image Diffusion Modelscited by 3
  9. 2023Responsible Emergent Multi-Agent Behaviorcited by 1
  10. 2024Mastering Zero-Shot Interactions in Cooperative and Competitive Simultaneous Gamescited by 1
  11. 2024Demystifying MuZero Planning: Interpreting the Learned Modelcited by 1
  12. 2025Improving Robustness of AlphaZero Algorithms to Test-Time Environment Changescited by 0
2 more
  1. 2025Relevance-Zone Reduction in Game Solvingcited by 0
  2. 2025A Study of Solving Life-and-Death Problems in Go Using Relevance-Zone Based Solverscited by 0