# Learning to Beat ByteRL: Exploitability of Collectible Card Game Agents

arXiv: https://arxiv.org/abs/2404.16689 · HTML: https://mlsanity.com/paper/2404.16689/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker](https://mlsanity.com/paper/1701.01724/) (2017)
- [Approximate exploitability: Learning a best response in large games](https://mlsanity.com/paper/2004.09677/) (2020)
- [Introducing the Hearthstone-AI Competition](https://mlsanity.com/paper/1906.04238/) (2019)
- [Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play](https://mlsanity.com/paper/2303.04096/) (2023)
- [Mastering Strategy Card Game (Hearthstone) with Improved Techniques](https://mlsanity.com/paper/2303.05197/) (2023)
- [Summarizing Strategy Card Game AI Competition](https://mlsanity.com/paper/2305.11814/) (2023)
- [Evolving Evaluation Functions for Collectible Card Game AI](https://mlsanity.com/paper/2105.01115/) (2021)
- [Helping AI to Play Hearthstone: AAIA'17 Data Mining Challenge](https://mlsanity.com/paper/1708.00730/) (2017)

## Built on by (most cited first, 1 of 1)

- [Unsound Search with Policy and Value Networks in Legends of Code and Magic](https://mlsanity.com/paper/2609.06816/) (2026)

## Abstract (arXiv)

While Poker, as a family of games, has been studied extensively in the last decades, collectible card games have seen relatively little attention. Only recently have we seen an agent that can compete with professional human players in Hearthstone, one of the most popular collectible card games. Although artificial agents must be able to work with imperfect information in both of these genres, collectible card games pose another set of distinct challenges. Unlike in many poker variants, agents must deal with state space so vast that even enumerating all states consistent with the agent's beliefs is intractable, rendering the current search methods unusable and requiring the agents to opt for other techniques. In this paper, we investigate the strength of such techniques for this class of games. Namely, we present preliminary analysis results of ByteRL, the state-of-the-art agent in Legends of Code and Magic and Hearthstone. Although ByteRL beat a top-10 Hearthstone player from China, we show that its play in Legends of Code and Magic is highly exploitable.