# Probabilistic Control and Majorization of Optimal Control

arXiv: https://arxiv.org/abs/2205.03279 · HTML: https://mlsanity.com/paper/2205.03279/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 46 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 8 of 8)

- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Maximum a Posteriori Policy Optimisation](https://mlsanity.com/paper/1806.06920/) (2018)
- [BADGR: An Autonomous Self-Supervised Learning-Based Navigation System](https://mlsanity.com/paper/2002.05700/) (2020)
- [Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving](https://mlsanity.com/paper/1707.02342/) (2017)
- [Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control](https://mlsanity.com/paper/1909.04947/) (2019)
- [Adaptive importance sampling for control and inference](https://mlsanity.com/paper/1505.01874/) (2015)
- [Entropy Regularised Deterministic Optimal Control: From Path Integral Solution to Sample-Based Trajectory Optimisation](https://mlsanity.com/paper/2110.02647/) (2021)

## Abstract (arXiv)

Probabilistic control design is founded on the principle that a rational agent attempts to match modelled with an arbitrary desired closed-loop system trajectory density. The framework was originally proposed as a tractable alternative to traditional optimal control design, parametrizing desired behaviour through fictitious transition and policy densities and using the information projection as a proximity measure. In this work we introduce an alternative parametrization of desired closed-loop behaviour and explore alternative proximity measures between densities. It is then illustrated how the associated probabilistic control problems solve into uncertain or probabilistic policies. Our main result is to show that the probabilistic control objectives majorize conventional, stochastic and risk sensitive, optimal control objectives. This observation allows us to identify two probabilistic fixed point iterations that converge to the deterministic optimal control policies establishing an explicit connection between either formulations. Further we demonstrate that the risk sensitive optimal control formulation is also technically equivalent to a Maximum Likelihood estimation problem on a probabilistic graph model where the notion of costs is directly encoded into the model. The associated treatment of the estimation problem is then shown to coincide with the moment projected probabilistic control formulation. That way optimal decision making can be reformulated as an iterative inference problem. Based on these insights we discuss directions for algorithmic development.