# Reasoning about Hypothetical Agent Behaviours and their Parameters

arXiv: https://arxiv.org/abs/1906.11064 · HTML: https://mlsanity.com/paper/1906.11064/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 24 of 24)

- [A Survey of Ad Hoc Teamwork Research](https://mlsanity.com/paper/2202.10450/) (2022)
- [COLA: Consistent Learning with Opponent-Learning Awareness](https://mlsanity.com/paper/2203.04098/) (2022)
- [Deep Interactive Bayesian Reinforcement Learning via Meta-Learning](https://mlsanity.com/paper/2101.03864/) (2021)
- [Expected Value of Communication for Planning in Ad Hoc Teamwork](https://mlsanity.com/paper/2103.01171/) (2021)
- [Generalization in Cooperative Multi-Agent Systems](https://mlsanity.com/paper/2202.00104/) (2022)
- [A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning](https://mlsanity.com/paper/2210.05448/) (2022)
- [LINDA: Multi-Agent Local Information Decomposition for Awareness of Teammates](https://mlsanity.com/paper/2109.12508/) (2021)
- [Who Needs to Know? Minimal Knowledge for Optimal Coordination](https://mlsanity.com/paper/2306.09309/) (2023)
- [(Ir)rationality in AI: State of the Art, Research Challenges and Open Questions](https://mlsanity.com/paper/2311.17165/) (2023)
- [An Extended Benchmarking of Multi-Agent Reinforcement Learning Algorithms in Complex Fully Cooperative Tasks](https://mlsanity.com/paper/2502.04773/) (2025)
- [Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration](https://mlsanity.com/paper/2103.04077/) (2021)
- [Inverse Attention Agents for Multi-Agent Systems](https://mlsanity.com/paper/2410.21794/) (2024)
- [Enhancing Cooperative Multi-Agent Reinforcement Learning with State Modelling and Adversarial Exploration](https://mlsanity.com/paper/2505.05262/) (2025)
- [Few-Shot Teamwork](https://mlsanity.com/paper/2207.09300/) (2022)
- [Decision-making with Speculative Opponent Models](https://mlsanity.com/paper/2211.11940/) (2022)
- [Presenting Multiagent Challenges in Team Sports Analytics](https://mlsanity.com/paper/2303.13660/) (2023)
- [On Diagnostics for Understanding Agent Training Behaviour in Cooperative MARL](https://mlsanity.com/paper/2312.08468/) (2023)
- [How much can change in a year? Revisiting Evaluation in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2312.08463/) (2023)
- [Seldonian Reinforcement Learning for Ad Hoc Teamwork](https://mlsanity.com/paper/2503.03885/) (2025)
- [Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork](https://mlsanity.com/paper/2607.27177/) (2026)
- [Combining a Meta-Policy and Monte-Carlo Planning for Scalable Type-Based Reasoning in Partially Observable Environments](https://mlsanity.com/paper/2306.06067/) (2023)
- [Preference-based opponent shaping in differentiable games](https://mlsanity.com/paper/2412.03072/) (2024)
- [LOD: Latent Objective Discovery in Heterogeneous Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2505.09756/) (2025)
- [NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming](https://mlsanity.com/paper/2602.17737/) (2026)

## Abstract (arXiv)

Agents can achieve effective interaction with previously unknown other agents by maintaining beliefs over a set of hypothetical behaviours, or types, that these agents may have. A current limitation in this method is that it does not recognise parameters within type specifications, because types are viewed as blackbox mappings from interaction histories to probability distributions over actions. In this work, we propose a general method which allows an agent to reason about both the relative likelihood of types and the values of any bounded continuous parameters within types. The method maintains individual parameter estimates for each type and selectively updates the estimates for some types after each observation. We propose different methods for the selection of types and the estimation of parameter values. The proposed methods are evaluated in detailed experiments, showing that updating the parameter estimates of a single type after each observation can be sufficient to achieve good performance.