# Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation

arXiv: https://arxiv.org/abs/2507.12186 · HTML: https://mlsanity.com/paper/2507.12186/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 25 references that are in the map.
- First posted on arXiv 16 Jul 2025.

## Builds on (most cited first, 4 of 4)

- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [Partially Observable Markov Decision Processes (POMDPs) and Robotics](https://mlsanity.com/paper/2107.07599/) (2021)
- [MAGIC: Learning Macro-Actions for Online POMDP Planning](https://mlsanity.com/paper/2011.03813/) (2020)
- [Scaling Long-Horizon Online POMDP Planning via Rapid State Space Sampling](https://mlsanity.com/paper/2411.07032/) (2024)

## Abstract (arXiv)

This paper proposes Partially Observable Reference Policy Programming, a novel anytime online approximate POMDP solver which samples meaningful future histories very deeply while simultaneously forcing a gradual policy update. We provide theoretical guarantees for the algorithm's underlying scheme which say that the performance loss is bounded by the average of the sampling approximation errors rather than the usual maximum, a crucial requirement given the sampling sparsity of online planning. Empirical evaluations on two large-scale problems with dynamically evolving environments -- including a helicopter emergency scenario in the Corsica region requiring approximately 150 planning steps -- corroborate the theoretical results and indicate that our solver considerably outperforms current online benchmarks.