# Policy Optimization for Markovian Jump Linear Quadratic Control: Gradient-Based Methods and Global Convergence

arXiv: https://arxiv.org/abs/2011.11852 · HTML: https://mlsanity.com/paper/2011.11852/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 4 of 4)

- [Convergence of Gradient-based MAML in LQR](https://mlsanity.com/paper/2309.06588/) (2023)
- [Global Convergence Using Policy Gradient Methods for Model-free Markovian Jump Linear Quadratic Control](https://mlsanity.com/paper/2111.15228/) (2021)
- [Certainty Equivalent Quadratic Control for Markov Jump Systems](https://mlsanity.com/paper/2105.12358/) (2021)
- [On the Optimization Landscape of Dynamic Output Feedback: A Case Study for Linear Quadratic Regulator](https://mlsanity.com/paper/2209.05042/) (2022)

## Abstract (arXiv)

Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. In this paper, we investigate the global convergence of gradient-based policy optimization methods for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS). First, we study the optimization landscape of direct policy optimization for MJLS, with static state feedback controllers and quadratic performance costs. Despite the non-convexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and almost smoothness. Based on these properties, we show global convergence of three types of policy optimization methods: the gradient descent method; the Gauss-Newton method; and the natural policy gradient method. We prove that all three methods converge to the optimal state feedback controller for MJLS at a linear rate if initialized at a controller which is mean-square stabilizing. Some numerical examples are presented to support the theory. This work brings new insights for understanding the performance of policy gradient methods on the Markovian jump linear quadratic control problem.