mlsanityData to 14 Oct 2026

A Generalist Hanabi Agent

arXiv 2503.14555 ↗First posted 17 Mar 2025cs.MA cs.AIcited by 5

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

Traditional multi-agent reinforcement learning (MARL) systems can develop cooperative strategies through repeated interactions. However, these systems are unable to perform well on any other setting than the one they have been trained on, and struggle to successfully cooperate with unfamiliar collaborators. This is particularly visible in the Hanabi benchmark, a popular 2-to-5 player cooperative card-game which requires complex reasoning and precise assistance to other agents. Current MARL agents for Hanabi can only learn one specific game-setting (e.g., 2-player games), and play with the same algorithmic agents. This is in stark contrast to humans, who can quickly adjust their strategies to work with unfamiliar partners or situations. In this paper, we introduce Recurrent Replay Relevance Distributed DQN (R3D2), a generalist agent for Hanabi, designed to overcome these limitations. We reformulate the task using text, as language has been shown to improve transfer. We then propose a distributed MARL algorithm that copes with the resulting dynamic observation- and action-space. In doing so, our agent is the first that can play all game settings concurrently, and extend strategies learned from one setting to other ones. As a consequence, our agent also demonstrates the ability to collaborate with different algorithmic agents -- agents that are themselves unable to do so. The implementation code is available at: hrefhttps://github.com/chandar-lab/R3D2-A-Generalist-Hanabi-AgentR3D2-A-Generalist-Hanabi-Agent

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 46 of its 64 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

46 of 64 references.

References · 46

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2020Scaling Laws for Neural Language Modelscited by 6,586
  7. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  10. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  11. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,167
  12. 2015Prioritized Experience Replaycited by 1,099
34 more
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  3. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  4. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  5. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  6. 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 327
  7. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  8. 2018Distributed Prioritized Experience Replaycited by 269
  9. 2020Experience Grounds Languagecited by 242
  10. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 231
  11. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 203
  12. 2020"Other-Play" for Zero-Shot Coordinationcited by 198
  13. 2019The Hanabi Challenge: A New Frontier for AI Researchcited by 197
  14. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  15. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  16. 2019Interactive Fiction Games: A Colossal Adventurecited by 168
  17. 2020Open Problems in Cooperative AIcited by 168
  18. 2020Keep CALM and Explore: Language Models for Action Generation in Text-based Gamescited by 108
  19. 2015Deep Reinforcement Learning with a Natural Language Action Spacecited by 88
  20. 2021Off-Belief Learningcited by 75
  21. 2018Bayesian Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 68
  22. 2019Simplified Action Decoder for Deep Multi-Agent Reinforcement Learningcited by 48
  23. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  24. 2021Continuous Coordination As a Realistic Scenario for Lifelong Learningcited by 32
  25. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 32
  26. 2023MAESTRO: Open-Ended Environment Design for Multi-Agent Reinforcement Learningcited by 31
  27. 2022Any-Play: An Intrinsic Augmentation for Zero-Shot Coordinationcited by 23
  28. 2021Pre-trained Language Models as Prior Knowledge for Playing Text-based Gamescited by 21
  29. 2020Enhancing Text-based Reinforcement Learning Agents with Commonsense Knowledgecited by 16
  30. 2019Algorithmic Improvements for Deep Reinforcement Learning applied to Interactive Fictioncited by 14
  31. 2021A Survey of Text Games for Reinforcement Learning informed by Natural Languagecited by 14
  32. 2023Towards Few-shot Coordination: Revisiting Ad-hoc Teamplay Challenge In the Game of Hanabicited by 10
  33. 2019Learn How to Cook a New Recipe in a New House: Using Map Familiarization, Curriculum Learning, and Bandit Feedback to Learn Families of Text-Based Adventure Gamescited by 7
  34. 2023Language Model-In-The-Loop: Data Optimal Approach to Learn-To-Recommend Actions in Text Gamescited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 3.

5 papers.

  1. 2025The Yokai Learning Environment: Tracking Beliefs Over Space and Timecited by 3
  2. 2025A Principle of Targeted Intervention for Multi-Agent Reinforcement Learningcited by 3
  3. 2026The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinatecited by 2
  4. 2026Learning Provably Correct Distributed Protocols Without Human Knowledgecited by 0
  5. 2026From Blueprint to Reality: Modeling and Applying Putnam's Social Capital Theory with LLM-based Multi-agent Simulationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MARL-LNS: Cooperative Multi-agent Reinforcement Learning via Large Neighborhoods Search
  2. 2024YOLO-MARL: You Only LLM Once for Multi-Agent Reinforcement Learning
  3. 2024Augmenting the action space with conventions to improve multi-agent cooperation in Hanabi
  4. 2025Reinforcement Learning for Hanabi
  5. 2026Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents
  6. 2021Reinforcement Learning In Two Player Zero Sum Simultaneous Action Games
  7. 2024FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning
  8. 2025Structured Cooperative Multi-Agent Reinforcement Learning: a Bayesian Network Perspective