# Safe Linear-Quadratic Dual Control with Almost Sure Performance Guarantee

arXiv: https://arxiv.org/abs/2103.13278 · HTML: https://mlsanity.com/paper/2103.13278/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 18 of 18)

- [On the Sample Complexity of the Linear Quadratic Regulator](https://mlsanity.com/paper/1710.01688/) (2017)
- [Naive Exploration is Optimal for Online LQR](https://mlsanity.com/paper/2001.09576/) (2020)
- [Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulator](https://mlsanity.com/paper/1805.09388/) (2018)
- [Non-asymptotic Identification of LTI Systems from a Single Trajectory](https://mlsanity.com/paper/1806.05722/) (2018)
- [Learning Linear-Quadratic Regulators Efficiently with only √T Regret](https://mlsanity.com/paper/1902.06223/) (2019)
- [Finite Time Identification in Unstable Linear Systems](https://mlsanity.com/paper/1710.01852/) (2017)
- [Non-asymptotic Identification of Linear Dynamical Systems Using Multiple Trajectories](https://mlsanity.com/paper/2009.00739/) (2020)
- [Logarithmic Regret for Learning Linear Quadratic Regulators Efficiently](https://mlsanity.com/paper/2002.08095/) (2020)
- [Active Learning for Identification of Linear Dynamical Systems](https://mlsanity.com/paper/2002.00495/) (2020)
- [On Adaptive Linear-Quadratic Regulators](https://mlsanity.com/paper/1806.10749/) (2018)
- [Input Perturbations for Adaptive Control and Learning](https://mlsanity.com/paper/1811.04258/) (2018)
- [Sample Complexity of Kalman Filtering for Unknown Systems](https://mlsanity.com/paper/1912.12309/) (2019)
- [Learning-based Control of Unknown Linear Systems with Thompson Sampling](https://mlsanity.com/paper/1709.04047/) (2017)
- [Finite Time Adaptive Stabilization of LQ Systems](https://mlsanity.com/paper/1807.09120/) (2018)
- [Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systems](https://mlsanity.com/paper/1905.06978/) (2019)
- [Finite-Data Performance Guarantees for the Output-Feedback Control of an Unknown System](https://mlsanity.com/paper/1803.09186/) (2018)
- [An Online Approach to Physical Watermark Design](https://mlsanity.com/paper/1911.01868/) (2019)
- [Optimistic robust linear quadratic dual control](https://mlsanity.com/paper/1912.13143/) (2019)

## Abstract (arXiv)

This paper considers the linear-quadratic dual control problem where the system parameters need to be identified and the control objective needs to be optimized in the meantime. Contrary to existing works on data-driven linear-quadratic regulation, which typically provide error or regret bounds within a certain probability, we propose an online algorithm that guarantees the asymptotic optimality of the controller in the almost sure sense. Our dual control strategy consists of two parts: a switched controller with time-decaying exploration noise and Markov parameter inference based on the cross-correlation between the exploration noise and system output. Central to the almost sure performance guarantee is a safe switched control strategy that falls back to a known conservative but stable controller when the actual state deviates significantly from the target state. We prove that this switching strategy rules out any potential destabilizing controllers from being applied, while the performance gap between our switching strategy and the optimal linear state feedback is exponentially small. Under our dual control scheme, the parameter inference error scales as O(T-1/4+ε), while the suboptimality gap of control performance scales as O(T-1/2+ε), where T is the number of time steps, and ε is an arbitrarily small positive number. Simulation results on an industrial process example are provided to illustrate the effectiveness of our proposed strategy.