# Entropy Maximization for Markov Decision Processes Under Temporal Logic Constraints

arXiv: https://arxiv.org/abs/1807.03223 · HTML: https://mlsanity.com/paper/1807.03223/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jul 2018.

## Built on by (most cited first, 13 of 13)

- [Fast Rates for Maximum Entropy Exploration](https://mlsanity.com/paper/2303.08059/) (2023)
- [Smoother Entropy for Active State Trajectory Estimation and Obfuscation in POMDPs](https://mlsanity.com/paper/2108.10227/) (2021)
- [Planning Not to Talk: Multiagent Systems that are Robust to Communication Loss](https://mlsanity.com/paper/2201.06619/) (2022)
- [Deceptive Planning for Resource Allocation](https://mlsanity.com/paper/2206.01306/) (2022)
- [Information-Theoretic Opacity-Enforcement in Markov Decision Processes](https://mlsanity.com/paper/2405.00157/) (2024)
- [TAB-Fields: A Maximum Entropy Framework for Mission-Aware Adversarial Planning](https://mlsanity.com/paper/2412.02570/) (2024)
- [Entropy-Regularized Partially Observed Markov Decision Processes](https://mlsanity.com/paper/2112.12255/) (2021)
- [Average-Reward Maximum Entropy Reinforcement Learning for Underactuated Double Pendulum Tasks](https://mlsanity.com/paper/2409.08938/) (2024)
- [Deceptive Sequential Decision-Making via Regularized Policy Optimization](https://mlsanity.com/paper/2501.18803/) (2025)
- [Policy Gradient Methods for Information-Theoretic Opacity in Markov Decision Processes](https://mlsanity.com/paper/2511.02704/) (2025)
- [Goal inference with Rao-Blackwellized Particle Filters](https://mlsanity.com/paper/2512.09269/) (2025)
- [Entropy Objectives in Markov Decision Processes](https://mlsanity.com/paper/2606.21726/) (2026)
- [Masking Radar Cognition under Adversarial Surveillance: A Distributional Privacy Framework](https://mlsanity.com/paper/2609.07428/) (2026)

## Abstract (arXiv)

We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes the exploration of different paths in an MDP while ensuring the satisfaction of a temporal logic specification. We first show that the maximum entropy of an MDP can be finite, infinite or unbounded. We provide necessary and sufficient conditions under which the maximum entropy of an MDP is finite, infinite or unbounded. We then present an algorithm which is based on a convex optimization problem to synthesize a policy that maximizes the entropy of an MDP. We also show that maximizing the entropy of an MDP is equivalent to maximizing the entropy of the paths that reach a certain set of states in the MDP. Finally, we extend the algorithm to an MDP subject to a temporal logic specification. In numerical examples, we demonstrate the proposed method on different motion planning scenarios and illustrate the relation between the restrictions imposed on the paths by a specification, the maximum entropy, and the predictability of paths.