# Deterministic Trajectory Optimization through Probabilistic Optimal Control

arXiv: https://arxiv.org/abs/2407.13316 · HTML: https://mlsanity.com/paper/2407.13316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 6 of 6)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Stochastic Optimal Control as Approximate Input Inference](https://mlsanity.com/paper/1910.03003/) (2019)
- [Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk](https://mlsanity.com/paper/2103.06319/) (2021)
- [Efficient Stochastic Optimal Control through Approximate Bayesian Input Inference](https://mlsanity.com/paper/2105.07693/) (2021)
- [Fourier--Hermite Dynamic Programming for Optimal Control](https://mlsanity.com/paper/2202.13453/) (2022)

## Abstract (arXiv)

In this article, we discuss two algorithms tailored to discrete-time deterministic finite-horizon nonlinear optimal control problems or so-called deterministic trajectory optimization problems. Both algorithms can be derived from an emerging theoretical paradigm that we refer to as probabilistic optimal control. The paradigm reformulates stochastic optimal control as an equivalent probabilistic inference problem and can be viewed as a generalisation of the former. The merit of this perspective is that it allows to address the problem using the Expectation-Maximization algorithm. It is shown that the application of this algorithm results in a fixed point iteration of probabilistic policies that converge to the deterministic optimal policy. Two strategies for policy evaluation are discussed, using state-of-the-art uncertainty quantification methods resulting into two distinct algorithms. The algorithms are structurally closest related to the differential dynamic programming algorithm and related methods that use sigma-point methods to avoid direct gradient evaluations. The main advantage of the algorithms is an improved balance between exploration and exploitation over the iterations, leading to improved numerical stability and accelerated convergence. These properties are demonstrated on different nonlinear systems.