mlsanityData to 14 Oct 2026

Self-Play Reinforcement Learning under Imperfect Information in Big 2

arXiv 2605.28863 ↗First posted 21 May 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Imperfect-information multiplayer games test whether agents can act under hidden information, sparse rewards, and non-stationary opponents. We study these challenges in Big 2, a four-player imperfect-information card game. We develop a self-play RL framework for Big 2 that enables controlled comparisons between policy-gradient and value-approximating agents. Under a common environment, input representation, training budget, and evaluation protocol, PPO outperforms Monte Carlo Q approximation, SARSA, and Q-learning against random, greedy, and heuristic Big 2 opponents. We further find that moderate entropy regularization improves PPO by preventing the policy from becoming overly deterministic, and that current-policy self-play provides a stronger finite-budget curriculum than checkpoint self-play or fixed-opponent training. Together, these results show that Big 2 is a useful controlled setting for studying deep RL under imperfect information, multiplayer interaction, delayed rewards, and variable action sets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 20 references that are in the map. First posted on arXiv 21 May 2026.

Builds on

12 of 20 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A Policy-Gradient Approach to Solving Imperfect-Information Games with Best-Iterate Convergence1 Aug
  2. 2024RL-CFR: Improving Action Abstraction for Imperfect Information Extensive-Form Games with Reinforcement Learning7 Mar

Other references · 10

  1. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  2. 2017DeepStack: Expert-Level Artificial Intelligence in No-Limit Pokercited by 233
  3. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  4. 2016Deep Reinforcement Learning from Self-Play in Imperfect-Information Gamescited by 158
  5. 2022Mastering the Game of Stratego with Model-Free Multiagent Reinforcement Learningcited by 131
  6. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 114
  7. 2018Deep Counterfactual Regret Minimizationcited by 103
  8. 2020Suphx: Mastering Mahjong with Deep Reinforcement Learningcited by 58
  9. 2021Student of Games: A unified learning algorithm for both perfect and imperfect information gamescited by 23
  10. 2018Application of Self-Play Reinforcement Learning to a Four-Player Game of Imperfect Informationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Reevaluating Policy Gradient Methods for Imperfect-Information Games
  2. 2026Data-Augmented Game Starts for Accelerating Self-Play Exploration in Imperfect Information Games
  3. 2026GAE Falls Short in Imperfect-Information Self-Play Reinforcement Learning
  4. 2026NashDreamer: Model-Based Reinforcement Learning for Zero-Sum Imperfect-Information Games
  5. 2024A Survey on Self-play Methods in Reinforcement Learning
  6. 2026Revisiting Regularized Policy Optimization for Stable and Efficient Reinforcement Learning in Two-Player Games
  7. 2024Improve Value Estimation of Q Function and Reshape Reward with Monte Carlo Tree Search
  8. 2026GameSmith: Scaling Zero-Sum Self-Play with Synthesised Games