# Entropy-Regularized Partially Observed Markov Decision Processes

arXiv: https://arxiv.org/abs/2112.12255 · HTML: https://mlsanity.com/paper/2112.12255/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 10 of 10)

- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints](https://mlsanity.com/paper/1807.03223/) (2018)
- [Rate-cost tradeoffs in control](https://mlsanity.com/paper/1612.02126/) (2016)
- [LQG Control with Minimum Directed Information: Semidefinite Programming Approach](https://mlsanity.com/paper/1510.04214/) (2015)
- [Rationally inattentive control of Markov processes](https://mlsanity.com/paper/1502.03762/) (2015)
- [Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPs](https://mlsanity.com/paper/2108.10227/) (2021)
- [Active Classification for POMDPs: a Kalman-like State Estimator](https://mlsanity.com/paper/1312.2039/) (2013)
- [Transfer-Entropy-Regularized Markov Decision Processes](https://mlsanity.com/paper/1708.09096/) (2017)
- [Directed Information on Abstract Spaces: Properties and Variational Equalities](https://mlsanity.com/paper/1302.3971/) (2013)
- [Active Trajectory Estimation for Partially Observed Markov Decision Processes via Conditional Entropy](https://mlsanity.com/paper/2104.01545/) (2021)

## Built on by (most cited first, 1 of 1)

- [Entropy-regularized Point-based Value Iteration](https://mlsanity.com/paper/2402.09388/) (2024)

## Abstract (arXiv)

We investigate partially observed Markov decision processes (POMDPs) with cost functions regularized by entropy terms describing state, observation, and control uncertainty. Standard POMDP techniques are shown to offer bounded-error solutions to these entropy-regularized POMDPs, with exact solutions possible when the regularization involves the joint entropy of the state, observation, and control trajectories. Our joint-entropy result is particularly surprising since it constitutes a novel, tractable formulation of active state estimation.