Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
This paper investigates a model-free solution to the stochastic linear quadratic regulation (LQR) problem for linear discrete-time systems with both multiplicative and additive noises. We formulate the stochastic LQR problem as a nonconvex optimization problem and rigorously analyze its dual problem structure. By exploiting the inherent convexity of the dual problem and analyzing Karush-Kuhn-Tucker conditions with respect to optimality in convex optimization, we establish an explicit relationship between the optimal point of the dual problem and the parameters of the associated Q-function. This theoretical insight, combined with the technique of the matrix direct sum, makes it possible to develop a novel model-free sample-efficient, non-iterative semidefinite programming algorithm that directly estimates optimal control gain without requiring an initial stabilizing controller, or noises measurability. The robustness of the model-free SDP method to errors is investigated. Our approach provides a new optimization-theoretic framework for understanding Q-learning algorithms while advancing the theoretical foundations of reinforcement learning in stochastic optimal control. Numerical validation on a pulse-width modulated inverter system demonstrates the algorithm's effectiveness, particularly in achieving a single-step non-iterative solution without hyper-parameter tuning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 25 references that are in the map. First posted on arXiv 24 Dec 2025.
Builds on
4 of 25 references.
References · 4
- 2021Convex optimizationcited by 2,515
- 2020Model-free optimal control of discrete-time systems with additive and multiplicative noisescited by 8
- 2021Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systemscited by 8
- 2020Robust Reinforcement Learning: A Case Study in Linear Quadratic Regulationcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Model-free stochastic linear quadratic design by semidefinite programming
- 2025Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise
- 2026Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching
- 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
- 2025A Fully Data-Driven Value Iteration for Stochastic LQR: Convergence, Robustness and Stability
- 2026(Cheap) Stochastic Policy Gradient Converges with High Probability for Linear Quadratic Regulator
- 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
- 2024Sample Complexity of the Linear Quadratic Regulator: A Reinforcement Learning Lens