# Entropy-regularized Point-based Value Iteration

arXiv: https://arxiv.org/abs/2402.09388 · HTML: https://mlsanity.com/paper/2402.09388/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Builds on (most cited first, 8 of 8)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [A Theory of Regularized Markov Decision Processes](https://mlsanity.com/paper/1901.11275/) (2019)
- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Task-Directed Exploration in Continuous POMDPs for Robotic Manipulation of Articulated Objects](https://mlsanity.com/paper/2212.04554/) (2022)
- [Entropy Maximization for Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2105.07490/) (2021)
- [Anderson Acceleration for Partially Observable Markov Decision Processes: A Maximum Entropy Approach](https://mlsanity.com/paper/2211.14998/) (2022)
- [Decomposition Methods with Deep Corrections for Reinforcement Learning](https://mlsanity.com/paper/1802.01772/) (2018)
- [Entropy-Regularized Partially Observed Markov Decision Processes](https://mlsanity.com/paper/2112.12255/) (2021)

## Built on by (most cited first, 1 of 1)

- [Convergence of regularized agent-state-based Q-learning in POMDPs](https://mlsanity.com/paper/2508.21314/) (2025)

## Abstract (arXiv)

Model-based planners for partially observable problems must accommodate both model uncertainty during planning and goal uncertainty during objective inference. However, model-based planners may be brittle under these types of uncertainty because they rely on an exact model and tend to commit to a single optimal behavior. Inspired by results in the model-free setting, we propose an entropy-regularized model-based planner for partially observable problems. Entropy regularization promotes policy robustness for planning and objective inference by encouraging policies to be no more committed to a single action than necessary. We evaluate the robustness and objective inference performance of entropy-regularized policies in three problem domains. Our results show that entropy-regularized policies outperform non-entropy-regularized baselines in terms of higher expected returns under modeling errors and higher accuracy during objective inference.