# Regret Analysis of Policy Optimization over Submanifolds for Linearly Constrained Online LQG

arXiv: https://arxiv.org/abs/2403.08553 · HTML: https://mlsanity.com/paper/2403.08553/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Optimization, Learning, and Games with Predictable Sequences](https://mlsanity.com/paper/1311.1869/) (2013)
- [Non-stationary Stochastic Optimization](https://mlsanity.com/paper/1307.5449/) (2013)
- [On the Sample Complexity of the Linear Quadratic Regulator](https://mlsanity.com/paper/1710.01688/) (2017)
- [Adaptive Online Learning in Dynamic Environments](https://mlsanity.com/paper/1810.10815/) (2018)
- [Naive Exploration is Optimal for Online LQR](https://mlsanity.com/paper/2001.09576/) (2020)
- [Online Control with Adversarial Disturbances](https://mlsanity.com/paper/1902.08721/) (2019)
- [Improper Learning for Non-Stochastic Control](https://mlsanity.com/paper/2001.09254/) (2020)
- [Online Linear Quadratic Control](https://mlsanity.com/paper/1806.07104/) (2018)
- [The Nonstochastic Control Problem](https://mlsanity.com/paper/1911.12178/) (2019)
- [Formulas for Data-driven Control: Stabilization, Optimality and Robustness](https://mlsanity.com/paper/1903.06842/) (2019)
- [Convergence and sample complexity of gradient methods for the model-free linear quadratic regulator problem](https://mlsanity.com/paper/1912.11899/) (2019)
- [LQR through the Lens of First Order Methods: Discrete-time Case](https://mlsanity.com/paper/1907.08921/) (2019)
- [Logarithmic Regret for Online Control](https://mlsanity.com/paper/1909.05062/) (2019)
- [Non-stationary Online Learning with Memory and Non-stochastic Control](https://mlsanity.com/paper/2102.03758/) (2021)
- [Online Optimal Control with Affine Constraints](https://mlsanity.com/paper/2010.04891/) (2020)
- [Policy Gradient-based Algorithms for Continuous-time Linear Quadratic Control](https://mlsanity.com/paper/2006.09178/) (2020)
- [Data-Enabled Policy Optimization for Direct Adaptive Learning of the LQR](https://mlsanity.com/paper/2401.14871/) (2024)
- [Low-complexity Learning of Linear Quadratic Regulators from Noisy Data](https://mlsanity.com/paper/2005.01082/) (2020)
- [Optimal Dynamic Regret in LQR Control](https://mlsanity.com/paper/2206.09257/) (2022)
- [Distributed Online Linear Quadratic Control for Linear Time-invariant Systems](https://mlsanity.com/paper/2009.13749/) (2020)
- [On the Certainty-Equivalence Approach to Direct Data-Driven LQR Design](https://mlsanity.com/paper/2109.06643/) (2021)
- [Policy Optimization over Submanifolds for Linearly Constrained Feedback Synthesis](https://mlsanity.com/paper/2201.11157/) (2022)
- [Dynamic Regret Minimization for Control of Non-stationary Linear Dynamical Systems](https://mlsanity.com/paper/2111.03772/) (2021)
- [Implications of Regret on Stability of Linear Dynamical Systems](https://mlsanity.com/paper/2211.07411/) (2022)
- [Learning Robust Data-based LQG Controllers from Noisy Data](https://mlsanity.com/paper/2305.01417/) (2023)
- [A Framework on Fully Distributed State Estimation and Cooperative Stabilization of LTI Plants](https://mlsanity.com/paper/2309.12081/) (2023)

## Abstract (arXiv)

Recent advancement in online optimization and control has provided novel tools to study online linear quadratic regulator (LQR) problems, where cost matrices are time-varying and unknown in advance. In this work, we study the online linear quadratic Gaussian (LQG) problem over the manifold of stabilizing controllers that are linearly constrained to impose physical conditions such as sparsity. By adopting a Riemannian perspective, we propose the online Newton on manifold (ONM) algorithm, which generates an online controller on-the-fly based on the second-order information of the cost function sequence. To quantify the algorithm performance, we use the notion of regret, defined as the sub-optimality of the algorithm cumulative cost against a (locally) minimizing controller sequence. We establish a regret bound in terms of the path-length of the benchmark minimizer sequence, and we further verify the effectiveness of ONM via simulations.