# Maximum Entropy Differential Dynamic Programming

arXiv: https://arxiv.org/abs/2110.06451 · HTML: https://mlsanity.com/paper/2110.06451/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2021.

## Built on by (most cited first, 5 of 5)

- [Maximum diffusion reinforcement learning](https://mlsanity.com/paper/2309.15293/) (2023)
- [Blending Data-Driven Priors in Dynamic Games](https://mlsanity.com/paper/2402.14174/) (2024)
- [Mutual Information Optimal Control of Discrete-Time Linear Systems](https://mlsanity.com/paper/2507.04712/) (2025)
- [Safe Beyond the Horizon: Efficient Sampling-based MPC with Neural Control Barrier Functions](https://mlsanity.com/paper/2502.15006/) (2025)
- [DDPEN: Trajectory Optimisation With Sub Goal Generation Model](https://mlsanity.com/paper/2301.07433/) (2023)

## Abstract (arXiv)

In this paper, we present a novel maximum entropy formulation of the Differential Dynamic Programming algorithm and derive two variants using unimodal and multimodal value functions parameterizations. By combining the maximum entropy Bellman equations with a particular approximation of the cost function, we are able to obtain a new formulation of Differential Dynamic Programming which is able to escape from local minima via exploration with a multimodal policy. To demonstrate the efficacy of the proposed algorithm, we provide experimental results using four systems on tasks that are represented by cost functions with multiple local minima and compare them against vanilla Differential Dynamic Programming. Furthermore, we discuss connections with previous work on the linearly solvable stochastic control framework and its extensions in relation to compositionality.