# Rethinking Formal Models of Partially Observable Multiagent Decision Making

arXiv: https://arxiv.org/abs/1906.11110 · HTML: https://mlsanity.com/paper/1906.11110/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2019.

## Built on by (most cited first, 30 of 36)

- [Near-Optimal Learning of Extensive-Form Games with Imperfect Information](https://mlsanity.com/paper/2202.01752/) (2022)
- [ESCHER: Eschewing Importance Sampling in Games by Computing a History Value Function to Estimate Regret](https://mlsanity.com/paper/2206.04122/) (2022)
- [Student of Games: A unified learning algorithm for both perfect and imperfect information games](https://mlsanity.com/paper/2112.03178/) (2021)
- [Causal Multi-Agent Reinforcement Learning: Review and Open Problems](https://mlsanity.com/paper/2111.06721/) (2021)
- [Solving Common-Payoff Games with Approximate Policy Iteration](https://mlsanity.com/paper/2101.04237/) (2021)
- [Illusory Attacks: Information-Theoretic Detectability Matters in Adversarial Attacks](https://mlsanity.com/paper/2207.10170/) (2022)
- [Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings](https://mlsanity.com/paper/2106.09086/) (2021)
- [Search in Imperfect Information Games](https://mlsanity.com/paper/2111.05884/) (2021)
- [HSVI can solve zero-sum Partially Observable Stochastic Games](https://mlsanity.com/paper/2210.14640/) (2022)
- [Detecting Multi-Agent Collusion Through Multi-Agent Interpretability](https://mlsanity.com/paper/2604.01151/) (2026)
- [Abstracting Imperfect Information Away from Two-Player Zero-Sum Games](https://mlsanity.com/paper/2301.09159/) (2023)
- [Strategy Synthesis for Zero-Sum Neuro-Symbolic Concurrent Stochastic Games](https://mlsanity.com/paper/2202.06255/) (2022)
- [Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation](https://mlsanity.com/paper/2405.19328/) (2024)
- [The Update-Equivalence Framework for Decision-Time Planning](https://mlsanity.com/paper/2304.13138/) (2023)
- [The Partially Observable History Process](https://mlsanity.com/paper/2111.08102/) (2021)
- [Learning not to Regret](https://mlsanity.com/paper/2303.01074/) (2023)
- [Partially Observable Stochastic Games with Neural Perception Mechanisms](https://mlsanity.com/paper/2310.11566/) (2023)
- [History Filtering in Imperfect Information Games: Algorithms and Complexity](https://mlsanity.com/paper/2311.14651/) (2023)
- [Solving Hierarchical Information-Sharing Dec-POMDPs: An Extensive-Form Game Approach](https://mlsanity.com/paper/2402.02954/) (2024)
- [Optimally Solving Simultaneous-Move Dec-POMDPs: The Sequential Central Planning Approach](https://mlsanity.com/paper/2408.13139/) (2024)
- [Neural Bayesian Filtering](https://mlsanity.com/paper/2510.03614/) (2025)
- [Look-ahead Reasoning with a Learned Model in Imperfect Information Games](https://mlsanity.com/paper/2510.05048/) (2025)
- [HSVI for zs-POSGs using Concavity, Convexity and Lipschitz Properties](https://mlsanity.com/paper/2110.14529/) (2021)
- [Self-Explaining Deviations for Coordination](https://mlsanity.com/paper/2207.12322/) (2022)
- [Mixture of Public and Private Distributions in Imperfect Information Games](https://mlsanity.com/paper/2405.14346/) (2024)
- [Approximating Nash Equilibria in General-Sum Games via Meta-Learning](https://mlsanity.com/paper/2504.18868/) (2025)
- [Online Competitive Information Gathering for Partially Observable Trajectory Games](https://mlsanity.com/paper/2506.01927/) (2025)
- [Modeling Uncertainty: Constraint-Based Belief States in Imperfect-Information Games](https://mlsanity.com/paper/2507.19263/) (2025)
- [Beyond Outcome-Based Imperfect-Recall: Higher-Resolution Abstractions for Imperfect-Information Games](https://mlsanity.com/paper/2510.15094/) (2025)
- [Revisiting Game Representations: The Hidden Costs of Efficiency in Sequential Decision-making Algorithms](https://mlsanity.com/paper/2112.10890/) (2021)

## Abstract (arXiv)

Multiagent decision-making in partially observable environments is usually modelled as either an extensive-form game (EFG) in game theory or a partially observable stochastic game (POSG) in multiagent reinforcement learning (MARL). One issue with the current situation is that while most practical problems can be modelled in both formalisms, the relationship of the two models is unclear, which hinders the transfer of ideas between the two communities. A second issue is that while EFGs have recently seen significant algorithmic progress, their classical formalization is unsuitable for efficient presentation of the underlying ideas, such as those around decomposition. To solve the first issue, we introduce factored-observation stochastic games (FOSGs), a minor modification of the POSG formalism which distinguishes between private and public observation and thereby greatly simplifies decomposition. To remedy the second issue, we show that FOSGs and POSGs are naturally connected to EFGs: by "unrolling" a FOSG into its tree form, we obtain an EFG. Conversely, any perfect-recall timeable EFG corresponds to some underlying FOSG in this manner. Moreover, this relationship justifies several minor modifications to the classical EFG formalization that recently appeared as an implicit response to the model's issues with decomposition. Finally, we illustrate the transfer of ideas between EFGs and MARL by presenting three key EFG techniques -- counterfactual regret minimization, sequence form, and decomposition -- in the FOSG framework.