Learning-based primal-dual optimal control of discrete-time stochastic systems with multiplicative noise
On the map: Deep reinforcement learning methods → RL for games and combinatorial optimization
Abstract (arXiv)
Reinforcement learning (RL) is an effective approach for solving optimal control problems without knowing the exact information of the system model. However, the classical Q-learning method, a model-free RL algorithm, has its limitations, such as lack of strict theoretical analysis and the need for artificial disturbances during implementation. This paper explores the partially model-free stochastic linear quadratic regulator (SLQR) problem for a system with multiplicative noise from the primal-dual perspective to address these challenges. This approach lays a strong theoretical foundation for understanding the intrinsic mechanisms of classical RL algorithms. We reformulate the SLQR into a non-convex primal-dual optimization problem and derive a strong duality result, which enables us to provide model-based and model-free algorithms for SLQR optimal policy design based on the Karush-Kuhn-Tucker (KKT) conditions. An illustrative example demonstrates the proposed model-free algorithm's validity, showcasing the central nervous system's learning mechanism in human arm movement.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for games and combinatorial optimization, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 39 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
11 of 39 references.
References · 11
- 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
- 2018The Gap Between Model-Based and Model-Free Methods on the Linear Quadratic Regulator: An Asymptotic Viewpointcited by 40
- 2019Finite-time Analysis of Approximate Policy Iteration for the Linear Quadratic Regulatorcited by 23
- 2018Model-Free Linear Quadratic Control via Reduction to Expert Predictioncited by 14
- 2020Can Primal Methods Outperform Primal-dual Methods in Decentralized Dynamic Optimization?cited by 11
- 2020Model-free optimal control of discrete-time systems with additive and multiplicative noisescited by 8
- 2021Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systemscited by 8
- 2023On stabilizability and exact observability of stochastic systems with their applicationscited by 5
- 2021Bounded Synthesis and Reinforcement Learning of Supervisors for Stochastic Discrete Event Systems with LTL Specificationscited by 2
- 2022Data-Driven Output Prediction and Control of Stochastic Systems: An Innovation-Based Approachcited by 2
- 2021Inverse linear-quadratic discrete-time finite-horizon optimal control for indistinguishable homogeneous agents: a convex optimization approachcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Model-free stochastic linear quadratic control for discrete-time systems with multiplicative and additive noises via semidefinite programming
- 2024Model-free stochastic linear quadratic design by semidefinite programming
- 2025Primal-dual policy learning for mean-field stochastic LQR problem ↗
- 2024Model-free optimal controller for discrete-time Markovian jump linear systems: A Q-learning approach
- 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
- 2025Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with Multiplicative Noise
- 2026Stabilizer Design for Policy Iteration in Stochastic Linear Quadratic Control: A Spectrum-Assignment Approach
- 2026Model-Free Q-Learning for Infinite-Horizon Stochastic Linear Quadratic Problems with Regime Switching