mlsanityData to 14 Oct 2026

Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model

arXiv 2511.07870 ↗First posted 11 Nov 2025eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization

Abstract (arXiv)

We study the problem of designing a state feedback linear quadratic Gaussian (LQG) controller for a system in which the system matrices as well as the process noise covariance are unknown. We do a rigorous comparison between two approaches. The first is the classic one in which a system identification stage is used to estimate the unknown parameters, which are then used in a state-feedback LQG (SF-LQG) controller design. The second approach is a recently proposed one using a reinforcement learning paradigm called Q-learning. We do the comparison in terms of complexity and accuracy of the resulting controller. We show that the classic approach asymptotically efficient, giving virtually no room for improvement in terms of accuracy. We also propose a novel Q-learning-based method which we show asymptotically achieves the optimal controller design. We complement our proposed method with a numerically efficient algorithmic implementation aiming at making it competitive in terms of computations. Nevertheless, our complexity analysis shows that the classic approach is still numerically more efficient than this Q-learning-based alternative. We then conclude that the classic approach remains being the best choice for addressing the SF-LQG design in the case of unknown parameters.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 27 references that are in the map. First posted on arXiv 11 Nov 2025.

Builds on

8 of 27 references.

References · 8

  1. 2018A Tour of Reinforcement Learning: The View from Continuous Controlcited by 151
  2. 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
  3. 2020Robust Policy Iteration for Continuous-time Linear Quadratic Regulationcited by 15
  4. 2020Model-free optimal control of discrete-time systems with additive and multiplicative noisescited by 8
  5. 2021Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systemscited by 8
  6. 2023Imitation and Transfer Learning for LQG Controlcited by 8
  7. 2022Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Controlcited by 7
  8. 2020Distributed Kalman Estimation with Decoupled Local Filterscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching
  2. 2026Cost-Driven Representation Learning for Linear Quadratic Gaussian Control: Part II
  3. 2024Data-Efficient Quadratic Q-Learning Using LMIs
  4. 2024Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach
  5. 2025On the Gradient Domination of the LQG Problem
  6. 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
  7. 2026On the Optimization Landscape of Observer-based Dynamic Linear Quadratic Control
  8. 2025Policy Gradient Method for LQG Control via Input-Output-History Representation: Convergence to O(ε)-Stationary Points