# On Adaptive Linear-Quadratic Regulators

arXiv: https://arxiv.org/abs/1806.10749 · HTML: https://mlsanity.com/paper/1806.10749/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2018.

## Built on by (most cited first, 21 of 21)

- [Joint Learning of Linear Time-Invariant Dynamical Systems](https://mlsanity.com/paper/2112.10955/) (2021)
- [Learning Mixtures of Linear Dynamical Systems](https://mlsanity.com/paper/2201.11211/) (2022)
- [Learning to Control Linear Systems can be Hard](https://mlsanity.com/paper/2205.14035/) (2022)
- [Analysis of Thompson Sampling for Partially Observable Contextual Multi-Armed Bandits](https://mlsanity.com/paper/2110.12175/) (2021)
- [Thompson Sampling Achieves tilde O(√T) Regret in Linear Quadratic Control](https://mlsanity.com/paper/2206.08520/) (2022)
- [Strong Consistency and Rate of Convergence of Switched Least Squares System Identification for Autonomous Markov Jump Linear Systems](https://mlsanity.com/paper/2112.10753/) (2021)
- [Augmented RBMLE-UCB Approach for Adaptive Control of Linear Quadratic Systems](https://mlsanity.com/paper/2201.10542/) (2022)
- [Finite-time System Identification and Adaptive Control in Autoregressive Exogenous Systems](https://mlsanity.com/paper/2108.11959/) (2021)
- [Joint Learning-Based Stabilization of Multiple Unknown Linear Systems](https://mlsanity.com/paper/2201.01387/) (2022)
- [Efficient Algorithms for Learning to Control Bandits with Unobserved Contexts](https://mlsanity.com/paper/2202.00867/) (2022)
- [Towards a Dimension-Free Understanding of Adaptive Linear Control](https://mlsanity.com/paper/2103.10620/) (2021)
- [Scalable regret for learning to control network-coupled subsystems with unknown dynamics](https://mlsanity.com/paper/2108.07970/) (2021)
- [Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems](https://mlsanity.com/paper/2112.15094/) (2021)
- [Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations](https://mlsanity.com/paper/2204.04773/) (2022)
- [Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces](https://mlsanity.com/paper/2603.08287/) (2026)
- [A relaxed technical assumption for posterior sampling-based reinforcement learning for control of unknown linear systems](https://mlsanity.com/paper/2108.08502/) (2021)
- [Safe Linear-Quadratic Dual Control with Almost Sure Performance Guarantee](https://mlsanity.com/paper/2103.13278/) (2021)
- [Reinforcement Learning Policies in Continuous-Time Linear Systems](https://mlsanity.com/paper/2109.07630/) (2021)
- [Regret Analysis of Certainty Equivalence Policies in Continuous-Time Linear-Quadratic Systems](https://mlsanity.com/paper/2206.04434/) (2022)
- [Almost Surely √T Regret for Adaptive LQR](https://mlsanity.com/paper/2301.05537/) (2023)
- [Finite Time Regret Bounds for Minimum Variance Control of Autoregressive Systems with Exogenous Inputs](https://mlsanity.com/paper/2305.16974/) (2023)

## Abstract (arXiv)

Performance of adaptive control policies is assessed through the regret with respect to the optimal regulator, which reflects the increase in the operating cost due to uncertainty about the dynamics parameters. However, available results in the literature do not provide a quantitative characterization of the effect of the unknown parameters on the regret. Further, there are problems regarding the efficient implementation of some of the existing adaptive policies. Finally, results regarding the accuracy with which the system's parameters are identified are scarce and rather incomplete. This study aims to comprehensively address these three issues. First, by introducing a novel decomposition of adaptive policies, we establish a sharp expression for the regret of an arbitrary policy in terms of the deviations from the optimal regulator. Second, we show that adaptive policies based on slight modifications of the Certainty Equivalence scheme are efficient. Specifically, we establish a regret of (nearly) square-root rate for two families of randomized adaptive policies. The presented regret bounds are obtained by using anti-concentration results on the random matrices employed for randomizing the estimates of the unknown parameters. Moreover, we study the minimal additional information on dynamics matrices that using them the regret will become of logarithmic order. Finally, the rates at which the unknown parameters of the system are being identified are presented.