mlsanityData to 7 Oct 2026

Mastering Strategy Card Game (Legends of Code and Magic) via End-to-End Policy and Optimistic Smooth Fictitious Play

arXiv 2303.04096 ↗First posted 7 Mar 2023cs.LG cs.AI cs.GTcited by 10

Not on the map. Why?

Abstract (arXiv)

Deep Reinforcement Learning combined with Fictitious Play shows impressive results on many benchmark games, most of which are, however, single-stage. In contrast, real-world decision making problems may consist of multiple stages, where the observation spaces and the action spaces can be completely different across stages. We study a two-stage strategy card game Legends of Code and Magic and propose an end-to-end policy to address the difficulties that arise in multi-stage game. We also propose an optimistic smooth fictitious play algorithm to find the Nash Equilibrium for the two-player game. Our approach wins double championships of COG2022 competition. Extensive studies verify and show the advancement of our approach.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 28 of its 54 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

28 of 54 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  5. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  6. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  7. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  8. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  9. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  10. 2018Recurrent World Models Facilitate Policy Evolutioncited by 761
  11. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  12. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
16 more
  1. 2016Safe and Efficient Off-Policy Reinforcement Learningcited by 260
  2. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 153
  3. 2015Fast Convergence of Regularized Learning in Gamescited by 149
  4. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 129
  5. 2017Cycles in adversarial regularized learningcited by 122
  6. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 110
  7. 2018Last-Iterate Convergence: Zero-Sum Games and Constrained Min-Max Optimizationcited by 97
  8. 2016Learning in games with continuous action sets and unknown payoff functionscited by 78
  9. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  10. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  11. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 55
  12. 2018Actor-Critic Policy Optimization in Partially Observable Multiagent Environmentscited by 35
  13. 2019Optimistic Regret Minimization for Extensive-Form Games via Dilated Distance-Generating Functionscited by 25
  14. 2020TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learningcited by 11
  15. 2021Search in Imperfect Information Gamescited by 8
  16. 2017Theoretical and Practical Advances on Smoothing for Extensive-Form Gamescited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 5; 2025: 1; 2026: 1.

10 papers.

  1. 2023Mastering Strategy Card Game (Hearthstone) with Improved Techniquescited by 10
  2. 2024Large Language Models as Agents in Two-Player Gamescited by 10
  3. 2024Contrastive Learning of Preferences with a Contextual InfoNCE Losscited by 9
  4. 2023Summarizing Strategy Card Game AI Competitioncited by 7
  5. 2024Learning With Generalised Card Representations for "Magic: The Gathering"cited by 3
  6. 2023Introducing Tales of Tribute AI Competitioncited by 2
  7. 2025UrzaGPT: LoRA-Tuned Large Language Models for Card Selection in Collectible Card Gamescited by 2
  8. 2024Learning to Beat ByteRL: Exploitability of Collectible Card Game Agentscited by 1
  9. 2024A Taxonomy of Collectible Card Games from a Game-Playing AI Perspectivecited by 0
  10. 2026Unsound Search with Policy and Value Networks in Legends of Code and Magiccited by 0