# Dynamic programming with incomplete information to overcome navigational uncertainty in a nautical environment

arXiv: https://arxiv.org/abs/2112.14657 · HTML: https://mlsanity.com/paper/2112.14657/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 29 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning](https://mlsanity.com/paper/2006.16908/) (2020)
- [Deep Variational Reinforcement Learning for POMDPs](https://mlsanity.com/paper/1806.02426/) (2018)
- [Reinforcement Learning of POMDPs using Spectral Methods](https://mlsanity.com/paper/1602.07764/) (2016)
- [4D-Net for Learned Multi-Modal Alignment](https://mlsanity.com/paper/2109.01066/) (2021)
- [Structured World Belief for Reinforcement Learning in POMDP](https://mlsanity.com/paper/2107.08577/) (2021)
- [Active Measure Reinforcement Learning for Observation Cost Minimization](https://mlsanity.com/paper/2005.12697/) (2020)
- [Reinforcement Learning for POMDP: Partitioned Rollout and Policy Iteration with Application to Autonomous Sequential Repair Problems](https://mlsanity.com/paper/2002.04175/) (2020)
- [Reinforcement Learning in POMDPs with Memoryless Options and Option-Observation Initiation Sets](https://mlsanity.com/paper/1708.06551/) (2017)
- [Optimizing thermodynamic trajectories using evolutionary and gradient-based reinforcement learning](https://mlsanity.com/paper/1903.08543/) (2019)
- [Convex Stochastic Dominance in Bayesian Localization, Filtering and Controlled Sensing POMDPs](https://mlsanity.com/paper/1904.00287/) (2019)
- [Controlled Sensing: A Myopic Fisher Information Sensor Selection Algorithm](https://mlsanity.com/paper/1408.4443/) (2014)
- [Dynamic Programming for POMDP with Jointly Discrete and Continuous State-Spaces](https://mlsanity.com/paper/1803.08876/) (2018)

## Built on by (most cited first, 1 of 1)

- [Optimizing Sensor Redundancy in Sequential Decision-Making Problems](https://mlsanity.com/paper/2412.07686/) (2024)

## Abstract (arXiv)

Using a novel toy nautical navigation environment, we show that dynamic programming can be used when only incomplete information about a partially observed Markov decision process (POMDP) is known. By incorporating uncertainty into our model, we show that navigation policies can be constructed that maintain safety, outperforming the baseline performance of traditional dynamic programming for Markov decision processes (MDPs). Adding in controlled sensing methods, we show that these policies can also lower measurement costs at the same time.