# On Convergence and Optimality of Best-Response Learning with Policy Types in Multiagent Systems

arXiv: https://arxiv.org/abs/1907.06995 · HTML: https://mlsanity.com/paper/1907.06995/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [Constant-Memory Strategies in Stochastic Games: Best Responses and Equilibria](https://mlsanity.com/paper/2505.07008/) (2025)
- [A Computable Game-Theoretic Framework for Multi-Agent Theory of Mind](https://mlsanity.com/paper/2511.22536/) (2025)
- [Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments](https://mlsanity.com/paper/2306.06067/) (2023)
- [Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference](https://mlsanity.com/paper/2610.07587/) (2026)

## Abstract (arXiv)

While many multiagent algorithms are designed for homogeneous systems (i.e. all agents are identical), there are important applications which require an agent to coordinate its actions without knowing a priori how the other agents behave. One method to make this problem feasible is to assume that the other agents draw their latent policy (or type) from a specific set, and that a domain expert could provide a specification of this set, albeit only a partially correct one. Algorithms have been proposed by several researchers to compute posterior beliefs over such policy libraries, which can then be used to determine optimal actions. In this paper, we provide theoretical guidance on two central design parameters of this method: Firstly, it is important that the user choose a posterior which can learn the true distribution of latent types, as otherwise suboptimal actions may be chosen. We analyse convergence properties of two existing posterior formulations and propose a new posterior which can learn correlated distributions. Secondly, since the types are provided by an expert, they may be inaccurate in the sense that they do not predict the agents' observed actions. We provide a novel characterisation of optimality which allows experts to use efficient model checking algorithms to verify optimality of types.