# Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model

arXiv: https://arxiv.org/abs/2511.07870 · HTML: https://mlsanity.com/paper/2511.07870/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods.
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 11 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL for games and combinatorial optimization](https://mlsanity.com/deeprl/rl-for-games-and-combinatorial-optimization/)

## Builds on (most cited first, 8 of 8)

- [A Tour of Reinforcement Learning: The View from Continuous Control](https://mlsanity.com/paper/1806.09460/) (2018)
- [Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulator](https://mlsanity.com/paper/1905.12842/) (2019)
- [Robust Policy Iteration for Continuous-time Linear Quadratic Regulation](https://mlsanity.com/paper/2005.09528/) (2020)
- [Model-free optimal control of discrete-time systems with additive and multiplicative noises](https://mlsanity.com/paper/2008.08734/) (2020)
- [Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems](https://mlsanity.com/paper/2107.07788/) (2021)
- [Imitation and Transfer Learning for LQG Control](https://mlsanity.com/paper/2303.09002/) (2023)
- [Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control](https://mlsanity.com/paper/2212.02072/) (2022)
- [Distributed Kalman Estimation with Decoupled Local Filters](https://mlsanity.com/paper/2009.05799/) (2020)

## Abstract (arXiv)

We study the problem of designing a state feedback linear quadratic Gaussian (LQG) controller for a system in which the system matrices as well as the process noise covariance are unknown. We do a rigorous comparison between two approaches. The first is the classic one in which a system identification stage is used to estimate the unknown parameters, which are then used in a state-feedback LQG (SF-LQG) controller design. The second approach is a recently proposed one using a reinforcement learning paradigm called Q-learning. We do the comparison in terms of complexity and accuracy of the resulting controller. We show that the classic approach asymptotically efficient, giving virtually no room for improvement in terms of accuracy. We also propose a novel Q-learning-based method which we show asymptotically achieves the optimal controller design. We complement our proposed method with a numerically efficient algorithmic implementation aiming at making it competitive in terms of computations. Nevertheless, our complexity analysis shows that the classic approach is still numerically more efficient than this Q-learning-based alternative. We then conclude that the classic approach remains being the best choice for addressing the SF-LQG design in the case of unknown parameters.