# A Dynamic Programming Algorithm for Finding an Optimal Sequence of Informative Measurements

arXiv: https://arxiv.org/abs/2109.11808 · HTML: https://mlsanity.com/paper/2109.11808/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 47 references that are in the map.
- First posted on arXiv 24 Sep 2021.

## Builds on (most cited first, 2 of 2)

- [Efficient Informative Sensing using Multiple Robots](https://mlsanity.com/paper/1401.3462/) (2014)
- [Multi-Robot Informative Path Planning for Active Sensing of Environmental Phenomena: A Tale of Two Algorithms](https://mlsanity.com/paper/1302.0723/) (2013)

## Built on by (most cited first, 3 of 3)

- [Applications of Reinforcement Learning in Finance -- Trading with a Double Deep Q-Network](https://mlsanity.com/paper/2206.14267/) (2022)
- [Learning controllable dynamics through informative exploration](https://mlsanity.com/paper/2507.06582/) (2025)
- [An Optimal Policy for Learning Controllable Dynamics by Exploration](https://mlsanity.com/paper/2512.20053/) (2025)

## Abstract (arXiv)

An informative measurement is the most efficient way to gain information about an unknown state. We present a first-principles derivation of a general-purpose dynamic programming algorithm that returns an optimal sequence of informative measurements by sequentially maximizing the entropy of possible measurement outcomes. This algorithm can be used by an autonomous agent or robot to decide where best to measure next, planning a path corresponding to an optimal sequence of informative measurements. The algorithm is applicable to states and controls that are either continuous or discrete, and agent dynamics that is either stochastic or deterministic; including Markov decision processes and Gaussian processes. Recent results from the fields of approximate dynamic programming and reinforcement learning, including on-line approximations such as rollout and Monte Carlo tree search, allow the measurement task to be solved in real time. The resulting solutions include non-myopic paths and measurement sequences that can generally outperform, sometimes substantially, commonly used greedy approaches. This is demonstrated for a global search task, where on-line planning for a sequence of local searches is found to reduce the number of measurements in the search by approximately half. A variant of the algorithm is derived for Gaussian processes for active sensing.