Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
This paper addresses infinite-horizon continuous-time stochastic linear quadratic optimal control problems with regime switching. We propose a paradigm shift from model-based design by adopting an adaptive dynamic programming approach, specifically developing on-policy and off-policy Q-learning algorithms that learn the optimal controller solely from online state trajectory data. The theoretical core of our work consists of a complete proof of the equivalence between the on- and off-policy architectures, alongside a rigorous analysis establishing the stability of the closed-loop system and the convergence of the algorithms to the optimal solution. For computational tractability, we implement these algorithms using vectorization and Kronecker product algebra. The theoretical results are corroborated by numerical case studies that clearly demonstrate the operational effectiveness and practical feasibility of the proposed model-free control strategy.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 3 of its 29 references that are in the map. First posted on arXiv 29 Jul 2026.
Builds on
3 of 29 references.
References · 3
- 2022q-Learning in Continuous Timecited by 91
- 2023Policy Iteration Reinforcement Learning Method for Continuous-Time Linear-Quadratic Mean-Field Control Problemscited by 4
- 2020A Q-learning algorithm for discrete-time linear-quadratic control with random parameters of unknown distribution: convergence and stabilizationcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
- 2024Model-free stochastic linear quadratic design by semidefinite programming
- 2024Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach
- 2025Comparative Study of Q-Learning for State-Feedback LQG Control with an Unknown Model
- 2025A model free approach for continuous-time optimal tracking control with unknown user-define cost and constrained control input via advantage function ↗
- 2025A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability
- 2025Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise
- 2025Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problems