# A Q-learning algorithm for discrete-time linear-quadratic control with random parameters of unknown distribution: convergence and stabilization

arXiv: https://arxiv.org/abs/2011.04970 · HTML: https://mlsanity.com/paper/2011.04970/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Learning Mixtures of Linear Dynamical Systems](https://mlsanity.com/paper/2201.11211/) (2022)
- [Policy Gradient Methods for Discrete Time Linear Quadratic Regulator With Random Parameters](https://mlsanity.com/paper/2303.16548/) (2023)
- [Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching](https://mlsanity.com/paper/2607.26456/) (2026)

## Abstract (arXiv)

This paper studies an infinite horizon optimal control problem for discrete-time linear systems and quadratic criteria, both with random parameters which are independent and identically distributed with respect to time. A classical approach is to solve an algebraic Riccati equation that involves mathematical expectations and requires certain statistical information of the parameters. In this paper, we propose an online iterative algorithm in the spirit of Q-learning for the situation where only one random sample of parameters emerges at each time step. The first theorem proves the equivalence of three properties: the convergence of the learning sequence, the well-posedness of the control problem, and the solvability of the algebraic Riccati equation. The second theorem shows that the adaptive feedback control in terms of the learning sequence stabilizes the system as long as the control problem is well-posed. Numerical examples are presented to illustrate our results.