mlsanityData to 14 Oct 2026

Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments

arXiv 2306.06067 ↗First posted 9 Jun 2023cs.AI cs.MAcited by 0

Not on the map. Why?

Abstract (arXiv)

The design of autonomous agents that can interact effectively with other agents without prior coordination is a core problem in multi-agent systems. Type-based reasoning methods achieve this by maintaining a belief over a set of potential behaviours for the other agents. However, current methods are limited in that they assume full observability of the state and actions of the other agent or do not scale efficiently to larger problems with longer planning horizons. Addressing these limitations, we propose Partially Observable Type-based Meta Monte-Carlo Planning (POTMMCP) - an online Monte-Carlo Tree Search based planning method for type-based reasoning in large partially observable environments. POTMMCP incorporates a novel meta-policy for guiding search and evaluating beliefs, allowing it to search more effectively to longer horizons using less planning time. We show that our method converges to the optimal solution in the limit and empirically demonstrate that it effectively adapts online to diverse sets of other agents across a range of environments. Comparisons with the state-of-the art method on problems with up to 1014 states and 108 observations indicate that POTMMCP is able to compute better solutions significantly faster.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 9 Jun 2023.

Builds on

19 of 47 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  4. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  5. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 131
  6. 2020Combining Deep Reinforcement Learning and Search for Imperfect-Information Gamescited by 114
  7. 2015A Game-Theoretic Model and Best-Response Learning Method for Ad Hoc Coordination in Multiagent Systemscited by 44
  8. 2019Improving Policies via Search in Cooperative Partially Observable Gamescited by 33
  9. 2022K-level Reasoning for Zero-Shot Coordination in Hanabicited by 32
  10. 2019Reasoning about Hypothetical Agent Behaviours and their Parameterscited by 24
  11. 2020Approximate exploitability: Learning a best response in large gamescited by 23
  12. 2015Belief and Truth in Hypothesised Behaviourscited by 21
7 more
  1. 2018Learning in POMDPs with Monte Carlo Tree Searchcited by 15
  2. 2014Scalable Planning and Learning for Multiagent POMDPs: Extended Versioncited by 14
  3. 2018Nested Reasoning About Autonomous Agents Using Probabilistic Programscited by 7
  4. 2019Scalable Decision-Theoretic Planning in Open and Typed Multiagent Systemscited by 7
  5. 2020Decentralized MCTS via Learned Teammate Modelscited by 6
  6. 2023Combining Tree-Search, Generative Models, and Nash Bargaining Concepts in Game-Theoretic Reinforcement Learningcited by 6
  7. 2019On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systemscited by 4

Built on by

Not cited yet.