# Efficient Planning under Uncertainty with Macro-actions

arXiv: https://arxiv.org/abs/1401.3827 · HTML: https://mlsanity.com/paper/1401.3827/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2014.

## Built on by (most cited first, 4 of 4)

- [Sequential Bayesian Optimization for Adaptive Informative Path Planning with Multimodal Sensing](https://mlsanity.com/paper/2209.07660/) (2022)
- [Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstraction](https://mlsanity.com/paper/2502.21186/) (2025)
- [When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions](https://mlsanity.com/paper/2406.07897/) (2024)
- [Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees](https://mlsanity.com/paper/2607.28399/) (2026)

## Abstract (arXiv)

Deciding how to act in partially observable environments remains an active area of research. Identifying good sequences of decisions is particularly challenging when good control performance requires planning multiple steps into the future in domains with many states. Towards addressing this challenge, we present an online, forward-search algorithm called the Posterior Belief Distribution (PBD). PBD leverages a novel method for calculating the posterior distribution over beliefs that result after a sequence of actions is taken, given the set of observation sequences that could be received during this process. This method allows us to efficiently evaluate the expected reward of a sequence of primitive actions, which we refer to as macro-actions. We present a formal analysis of our approach, and examine its performance on two very large simulation experiments: scientific exploration and a target monitoring domain. We also demonstrate our algorithm being used to control a real robotic helicopter in a target monitoring experiment, which suggests that our approach has practical potential for planning in real-world, large partially observable domains where a multi-step lookahead is required to achieve good performance.