# HSVI-based Online Minimax Strategies for Partially Observable Stochastic Games with Neural Perception Mechanisms

arXiv: https://arxiv.org/abs/2404.10679 · HTML: https://mlsanity.com/paper/2404.10679/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 12 of 12)

- [A Unified Game-Theoretic Approach to Multiagent Reinforcement Learning](https://mlsanity.com/paper/1711.00832/) (2017)
- [DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker](https://mlsanity.com/paper/1701.01724/) (2017)
- [Combining Deep Reinforcement Learning and Search for Imperfect-Information Games](https://mlsanity.com/paper/2007.13544/) (2020)
- [Rethinking Formal Models of Partially Observable Multiagent Decision Making](https://mlsanity.com/paper/1906.11110/) (2019)
- [Solving Imperfect Information Games Using Decomposition](https://mlsanity.com/paper/1303.4441/) (2013)
- [Student of Games: A unified learning algorithm for both perfect and imperfect information games](https://mlsanity.com/paper/2112.03178/) (2021)
- [Solving Zero-Sum One-Sided Partially Observable Stochastic Games](https://mlsanity.com/paper/2010.11243/) (2020)
- [HSVI can solve zero-sum Partially Observable Stochastic Games](https://mlsanity.com/paper/2210.14640/) (2022)
- [Monte Carlo Continual Resolving for Online Strategy Computation in Imperfect Information Games](https://mlsanity.com/paper/1812.07351/) (2018)
- [The Stackelberg Equilibrium for One-sided Zero-sum Partially Observable Stochastic Games](https://mlsanity.com/paper/2109.08339/) (2021)
- [Strategy Synthesis for Zero-Sum Neuro-Symbolic Concurrent Stochastic Games](https://mlsanity.com/paper/2202.06255/) (2022)
- [Partially Observable Stochastic Games with Neural Perception Mechanisms](https://mlsanity.com/paper/2310.11566/) (2023)

## Abstract (arXiv)

We consider a variant of continuous-state partially-observable stochastic games with neural perception mechanisms and an asymmetric information structure. One agent has partial information, with the observation function implemented as a neural network, while the other agent is assumed to have full knowledge of the state. We present, for the first time, an efficient online method to compute an varepsilon-minimax strategy profile, which requires only one linear program to be solved for each agent at every stage, instead of a complex estimation of opponent counterfactual values. For the partially-informed agent, we propose a continual resolving approach which uses lower bounds, pre-computed offline with heuristic search value iteration (HSVI), instead of opponent counterfactual values. This inherits the soundness of continual resolving at the cost of pre-computing the bound. For the fully-informed agent, we propose an inferred-belief strategy, where the agent maintains an inferred belief about the belief of the partially-informed agent based on (offline) upper bounds from HSVI, guaranteeing varepsilon-distance to the value of the game at the initial belief known to both agents.