# Scalable Solution Methods for Dec-POMDPs with Deterministic Dynamics

arXiv: https://arxiv.org/abs/2508.21595 · HTML: https://mlsanity.com/paper/2508.21595/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 29 Aug 2025.

## Builds on (most cited first, 8 of 8)

- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [The Complexity of Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1301.3836/) (2013)
- [Policy Iteration for Decentralized Control of Markov Decision Processes](https://mlsanity.com/paper/1401.3460/) (2014)
- [Solving infinite-horizon Dec-POMDPs using Finite State Controllers within JESP](https://mlsanity.com/paper/2109.08755/) (2021)
- [A Finite-State Controller Based Offline Solver for Deterministic POMDPs](https://mlsanity.com/paper/2505.00596/) (2025)
- [Monte-Carlo Search for an Equilibrium in Dec-POMDPs](https://mlsanity.com/paper/2305.11811/) (2023)

## Abstract (arXiv)

Many high-level multi-agent planning problems, including multi-robot navigation and path planning, can be effectively modeled using deterministic actions and observations. In this work, we focus on such domains and introduce the class of Deterministic Decentralized POMDPs (Det-Dec-POMDPs). This is a subclass of Dec-POMDPs characterized by deterministic transitions and observations conditioned on the state and joint actions. We then propose a practical solver called Iterative Deterministic POMDP Planning (IDPP). This method builds on the classic Joint Equilibrium Search for Policies framework and is specifically optimized to handle large-scale Det-Dec-POMDPs that current Dec-POMDP solvers are unable to address efficiently.