mlsanityData to 7 Oct 2026

Mastering Strategy Card Game (Hearthstone) with Improved Techniques

arXiv 2303.05197 ↗First posted 9 Mar 2023cs.LG cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

Strategy card game is a well-known genre that is demanding on the intelligent game-play and can be an ideal test-bench for AI. Previous work combines an end-to-end policy function and an optimistic smooth fictitious play, which shows promising performances on the strategy card game Legend of Code and Magic. In this work, we apply such algorithms to Hearthstone, a famous commercial game that is more complicated in game rules and mechanisms. We further propose several improved techniques and consequently achieve significant progress. For a machine-vs-human test we invite a Hearthstone streamer whose best rank was top 10 of the official league in China region that is estimated to be of millions of players. Our models defeat the human player in all Best-of-5 tournaments of full games (including both deck building and battle), showing a strong capability of decision making.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 9 Mar 2023.

Builds on

18 of 43 references.

References · 18

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  3. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  4. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  6. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
  7. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  8. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  9. 2018Deep Counterfactual Regret Minimizationcited by 102
  10. 2016Learning in games with continuous action sets and unknown payoff functionscited by 78
  11. 2022A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Gamescited by 64
  12. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
6 more
  1. 2018Unsupervised Predictive Memory in a Goal-Directed Agentcited by 54
  2. 2021Last-iterate Convergence in Extensive-Form Gamescited by 27
  3. 2018Improving Hearthstone AI by Combining MCTS and Supervised Learning Algorithmscited by 13
  4. 2020TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learningcited by 11
  5. 2019Introducing the Hearthstone-AI Competitioncited by 10
  6. 2023Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Playcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 1; 2026: 2.

10 papers.

  1. 2024Contrastive Learning of Preferences with a Contextual InfoNCE Losscited by 9
  2. 2023Summarizing Strategy Card Game AI Competitioncited by 7
  3. 2024Learning With Generalised Card Representations for "Magic: The Gathering"cited by 3
  4. 2023Introducing Tales of Tribute AI Competitioncited by 2
  5. 2025UrzaGPT: LoRA-Tuned Large Language Models for Card Selection in Collectible Card Gamescited by 2
  6. 2024Learning to Beat ByteRL: Exploitability of Collectible Card Game Agentscited by 1
  7. 2024Neural Network-based Information Set Weighting for Playing Reconnaissance Blind Chesscited by 1
  8. 2026AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agentscited by 1
  9. 2026Predicting Drafted Deck Strength for "Magic: the Gathering"cited by 1
  10. 2024A Taxonomy of Collectible Card Games from a Game-Playing AI Perspectivecited by 0