# Heuristics for Partially Observable Stochastic Contingent Planning

arXiv: https://arxiv.org/abs/2410.05870 · HTML: https://mlsanity.com/paper/2410.05870/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 38 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 4 of 4)

- [DESPOT: Online POMDP Planning with Regularization](https://mlsanity.com/paper/1609.03250/) (2016)
- [On the Sample Complexity of Actor-Critic Method for Reinforcement Learning with Function Approximation](https://mlsanity.com/paper/1910.08412/) (2019)
- [Factored Probabilistic Belief Tracking](https://mlsanity.com/paper/1909.13779/) (2019)
- [Rollout Heuristics for Online Stochastic Contingent Planning](https://mlsanity.com/paper/2310.02345/) (2023)

## Built on by (most cited first, 1 of 1)

- [Active Thinking Model: A Goal-Directed Self-Improving Framework for Real-World Adaptive Intelligence](https://mlsanity.com/paper/2511.00758/) (2025)

## Abstract (arXiv)

Acting to complete tasks in stochastic partially observable domains is an important problem in artificial intelligence, and is often formulated as a goal-based POMDP. Goal-based POMDPs can be solved using the RTDP-BEL algorithm, that operates by running forward trajectories from the initial belief to the goal. These trajectories can be guided by a heuristic, and more accurate heuristics can result in significantly faster convergence. In this paper, we develop a heuristic function that leverages the structured representation of domain models. We compute, in a relaxed space, a plan to achieve the goal, while taking into account the value of information, as well as the stochastic effects. We provide experiments showing that while our heuristic is slower to compute, it requires an order of magnitude less trajectories before convergence. Overall, it thus speeds up RTDP-BEL, particularly in problems where significant information gathering is needed.