# A Finite-State Controller Based Offline Solver for Deterministic POMDPs

arXiv: https://arxiv.org/abs/2505.00596 · HTML: https://mlsanity.com/paper/2505.00596/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 42 references that are in the map.
- First posted on arXiv 1 May 2025.

## Builds on (most cited first, 8 of 8)

- [Multiple-Environment Markov Decision Processes](https://mlsanity.com/paper/1405.4733/) (2014)
- [Optimal Cost Almost-sure Reachability in POMDPs](https://mlsanity.com/paper/1411.3880/) (2014)
- [POMDP-lite for Robust Robot Planning under Uncertainty](https://mlsanity.com/paper/1602.04875/) (2016)
- [Inductive Synthesis of Finite-State Controllers for POMDPs](https://mlsanity.com/paper/2203.10803/) (2022)
- [Long-Term Robot Navigation in Indoor Environments Estimating Patterns in Traversability Changes](https://mlsanity.com/paper/1909.12733/) (2019)
- [On Anderson acceleration for partially observable Markov decision processes](https://mlsanity.com/paper/2103.15275/) (2021)
- [LAMP: Learning a Motion Policy to Repeatedly Navigate in an Uncertain Environment](https://mlsanity.com/paper/2012.02271/) (2020)
- [Stochastic Planning for ASV Navigation Using Satellite Images](https://mlsanity.com/paper/2209.11864/) (2022)

## Built on by (most cited first, 2 of 2)

- [Scalable Solution Methods for Dec-POMDPs with Deterministic Dynamics](https://mlsanity.com/paper/2508.21595/) (2025)
- [Robustness Analysis of POMDP Policies to Observation Perturbations](https://mlsanity.com/paper/2604.21256/) (2026)

## Abstract (arXiv)

Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and observe deterministically. In this paper, we propose DetMCVI, an adaptation of the Monte Carlo Value Iteration (MCVI) algorithm for DetPOMDPs, which builds policies in the form of finite-state controllers (FSCs). DetMCVI solves large problems with a high success rate, outperforming existing baselines for DetPOMDPs. We also verify the performance of the algorithm in a real-world mobile robot forest mapping scenario.