mlsanityData to 14 Oct 2026

Diversifying AI: Towards Creative Chess with AlphaZero

arXiv 2308.09175 ↗First posted 17 Aug 2023cs.AI cs.LGcited by 15

Not on the map. Why?

Abstract (arXiv)

In recent years, Artificial Intelligence (AI) systems have surpassed human intelligence in a variety of computational tasks. However, AI systems, like humans, make mistakes, have blind spots, hallucinate, and struggle to generalize to new situations. This work explores whether AI can benefit from creative decision-making mechanisms when pushed to the limits of its computational rationality. In particular, we investigate whether a team of diverse AI systems can outperform a single AI in challenging tasks by generating more ideas as a group and then selecting the best ones. We study this question in the game of chess, the so-called drosophila of AI. We build on AlphaZero (AZ) and extend it to represent a league of agents via a latent-conditioned architecture, which we call AZ_db. We train AZ_db to generate a wider range of ideas using behavioral diversity techniques and select the most promising ones with sub-additive planning. Our experiments suggest that AZ_db plays chess in diverse ways, solves more puzzles as a group and outperforms a more homogeneous team. Notably, AZ_db solves twice as many challenging puzzles as AZ, including the challenging Penrose positions. When playing chess from different openings, we notice that players in AZ_db specialize in different openings, and that selecting a player for each opening using sub-additive planning results in a 50 Elo improvement over AZ. Our findings suggest that diversity bonuses emerge in teams of AI agents, just as they do in teams of humans and that diversity is a valuable asset in solving computationally hard problems.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 56 of its 101 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

56 of 101 references.

References · 56

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  5. 2015Illuminating search spaces by mapping elitescited by 425
  6. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 384
  7. 2017Population Based Training of Neural Networkscited by 367
  8. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
  10. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  11. 2016Variational Intrinsic Controlcited by 267
  12. 2018Provably Efficient Maximum Entropy Explorationcited by 242
38 more
  1. 2020Never Give Up: Learning Directed Exploration Strategiescited by 212
  2. 2014Robots that can adapt like animalscited by 204
  3. 2021Acquisition of Chess Knowledge in AlphaZerocited by 123
  4. 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
  5. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 113
  6. 2020Variational Policy Gradient Method for Reinforcement Learning with General Utilitiescited by 111
  7. 2019Transfer in Deep Reinforcement Learning Using Successor Features and Generalised Policy Improvementcited by 88
  8. 2021Reward is enough for convex MDPscited by 84
  9. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  10. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  11. 2016Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithmcited by 66
  12. 2020Real World Games Look Like Spinning Topscited by 66
  13. 2018Diversity-Driven Exploration Strategy for Deep Reinforcement Learningcited by 65
  14. 2019Autonomous skill discovery with Quality-Diversity and Unsupervised Descriptorscited by 57
  15. 2021Concave Utility Reinforcement Learning: the Mean-Field Game Viewpointcited by 56
  16. 2020Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimizationcited by 54
  17. 2020A Boolean Task Algebra for Reinforcement Learningcited by 48
  18. 2021Adversarially Guided Actor-Criticcited by 47
  19. 2017Stein Variational Policy Gradientcited by 44
  20. 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
  21. 2022Approximating Gradients for Differentiable Quality Diversity in Reinforcement Learningcited by 38
  22. 2022Discovering Policies with DOMiNO: Diversity Optimization Maintaining Near Optimalitycited by 37
  23. 2019Diversity-Inducing Policy Gradient: Using Maximum Mean Discrepancy to Find a Set of Diverse Policiescited by 34
  24. 2020Relative Variational Intrinsic Controlcited by 34
  25. 2022Challenging Common Assumptions in Convex Reinforcement Learningcited by 30
  26. 2021Online Apprenticeship Learningcited by 29
  27. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 29
  28. 2022MuZero with Self-competition for Rate Control in VP9 Video Compressioncited by 28
  29. 2023Superhuman Artificial Intelligence Can Improve Human Decision Making by Increasing Noveltycited by 27
  30. 2020Approximate exploitability: Learning a best response in large gamescited by 23
  31. 2021Discovering a set of policies for the worst case rewardcited by 21
  32. 2022Optimistic PAC Reinforcement Learning: the Instance-Dependent Viewcited by 17
  33. 2023Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applicationscited by 16
  34. 2020Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chesscited by 15
  35. 2021Constructing a Good Behavior Basis for Transfer using Generalized Policy Updatescited by 14
  36. 2022NeuPL: Neural Population Learningcited by 14
  37. 2019Apprenticeship Learning via Frank-Wolfecited by 13
  38. 2020Non-local Policy Optimization via Diversity-regularized Collaborative Explorationcited by 13

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 9; 2025: 2.

15 papers.

Sort:

By area:

  1. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 102
  2. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 42
  3. 2024Maia-2: A Unified Model for Human-AI Alignment in Chesscited by 25
  4. 2024Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Criticscited by 13
  5. 2024Policy Space Response Oracles: A Surveycited by 10
  6. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunitiescited by 7
  7. 2024Multi-Objective Quality-Diversity for Crystal Structure Predictioncited by 5
  8. 2024Predictive representations: building blocks of intelligencecited by 4
  9. 2024BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Modelscited by 4
  10. 2024Functional Acceleration for Policy Mirror Descentcited by 3
  11. 2024Convex Markov Games: A New Frontier for Multi-Agent Reinforcement Learningcited by 3
  12. 2023Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalizationcited by 2
3 more
  1. 2023Iterative Option Discovery for Planning, by Planningcited by 1
  2. 2024Preference-Conditioned Gradient Variations for Multi-Objective Quality-Diversitycited by 1
  3. 2025Limitations on Accurate, Trusted, Human-level Reasoningcited by 0