# Stochastic Optimal Control for Multivariable Dynamical Systems Using Expectation Maximization

arXiv: https://arxiv.org/abs/2010.00207 · HTML: https://mlsanity.com/paper/2010.00207/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 1 of 1)

- [Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States](https://mlsanity.com/paper/2210.00211/) (2022)

## Abstract (arXiv)

Trajectory optimization is a fundamental stochastic optimal control problem. This paper deals with a trajectory optimization approach for dynamical systems subject to measurement noise that can be fitted into linear time-varying stochastic models. Exact/complete solutions to these kind of control problems have been deemed analytically intractable in literature because they come under the category of Partially Observable Markov Decision Processes (POMDPs). Therefore, effective solutions with reasonable approximations are widely sought for. We propose a reformulation of stochastic control in a reinforcement learning setting. This type of formulation assimilates the benefits of conventional optimal control procedure, with the advantages of maximum likelihood approaches. Finally, an iterative trajectory optimization paradigm called as Stochastic Optimal Control - Expectation Maximization (SOC-EM) is put-forth. This trajectory optimization procedure exhibits better performance in terms of reduction of cumulative cost-to-go which is proved both theoretically and empirically. Furthermore, we also provide novel theoretical work which is related to uniqueness of control parameter estimates. Analysis of the control covariance matrix is presented, which handles stochasticity through efficiently balancing exploration and exploitation.