# A relaxed technical assumption for posterior sampling-based reinforcement learning for control of unknown linear systems

arXiv: https://arxiv.org/abs/2108.08502 · HTML: https://mlsanity.com/paper/2108.08502/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 9 of 9)

- [Naive Exploration is Optimal for Online LQR](https://mlsanity.com/paper/2001.09576/) (2020)
- [Regret Bounds for Robust Adaptive Control of the Linear Quadratic Regulator](https://mlsanity.com/paper/1805.09388/) (2018)
- [Learning Linear-Quadratic Regulators Efficiently with only √T Regret](https://mlsanity.com/paper/1902.06223/) (2019)
- [On Adaptive Linear-Quadratic Regulators](https://mlsanity.com/paper/1806.10749/) (2018)
- [Input Perturbations for Adaptive Control and Learning](https://mlsanity.com/paper/1811.04258/) (2018)
- [Efficient Optimistic Exploration in Linear-Quadratic Regulators via Lagrangian Relaxation](https://mlsanity.com/paper/2007.06482/) (2020)
- [Certainty Equivalence is Efficient for Linear Quadratic Control](https://mlsanity.com/paper/1902.07826/) (2019)
- [Bayesian Optimal Control of Smoothly Parameterized Systems: The Lazy Posterior Sampling Algorithm](https://mlsanity.com/paper/1406.3926/) (2014)
- [Thompson sampling for linear quadratic mean-field teams](https://mlsanity.com/paper/2011.04686/) (2020)

## Built on by (most cited first, 1 of 1)

- [Reinforcement Learning Policies in Continuous-Time Linear Systems](https://mlsanity.com/paper/2109.07630/) (2021)

## Abstract (arXiv)

We revisit the Thompson sampling algorithm to control an unknown linear quadratic (LQ) system recently proposed by Ouyang et al (arXiv:1709.04047). The regret bound of the algorithm was derived under a technical assumption on the induced norm of the closed loop system. In this technical note, we show that by making a minor modification in the algorithm (in particular, ensuring that an episode does not end too soon), this technical assumption on the induced norm can be replaced by a milder assumption in terms of the spectral radius of the closed loop system. The modified algorithm has the same Bayesian regret of tildeO(√T), where T is the time-horizon and the tildeO(·) notation hides logarithmic terms in~T.