# Regret Analysis of Certainty Equivalence Policies in Continuous-Time Linear-Quadratic Systems

arXiv: https://arxiv.org/abs/2206.04434 · HTML: https://mlsanity.com/paper/2206.04434/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 39 references that are in the map.
- First posted on arXiv 9 Jun 2022.

## Builds on (most cited first, 10 of 10)

- [Logarithmic Regret Bound in Partially Observable Linear Dynamical Systems](https://mlsanity.com/paper/2003.11227/) (2020)
- [Black-Box Control for Linear Dynamical Systems](https://mlsanity.com/paper/2007.06650/) (2020)
- [Logarithmic Regret for Learning Linear Quadratic Regulators Efficiently](https://mlsanity.com/paper/2002.08095/) (2020)
- [Logarithmic regret for episodic continuous-time linear-quadratic reinforcement learning over a finite-time horizon](https://mlsanity.com/paper/2006.15316/) (2020)
- [On Adaptive Linear-Quadratic Regulators](https://mlsanity.com/paper/1806.10749/) (2018)
- [Input Perturbations for Adaptive Control and Learning](https://mlsanity.com/paper/1811.04258/) (2018)
- [On Applications of Bootstrap in Continuous Space Reinforcement Learning](https://mlsanity.com/paper/1903.05803/) (2019)
- [On Uninformative Optimal Policies in Adaptive LQR with Unknown B-Matrix](https://mlsanity.com/paper/2011.09288/) (2020)
- [Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systems](https://mlsanity.com/paper/1905.06978/) (2019)
- [Bayesian Algorithms Learn to Stabilize Unknown Continuous-Time Systems](https://mlsanity.com/paper/2112.15094/) (2021)

## Abstract (arXiv)

This work theoretically studies a ubiquitous reinforcement learning policy for controlling the canonical model of continuous-time stochastic linear-quadratic systems. We show that randomized certainty equivalent policy addresses the exploration-exploitation dilemma in linear control systems that evolve according to unknown stochastic differential equations and their operating cost is quadratic. More precisely, we establish square-root of time regret bounds, indicating that randomized certainty equivalent policy learns optimal control actions fast from a single state trajectory. Further, linear scaling of the regret with the number of parameters is shown. The presented analysis introduces novel and useful technical approaches, and sheds light on fundamental challenges of continuous-time reinforcement learning.