mlsanityData to 14 Oct 2026

Global Convergence of Policy Gradient for Entropy Regularized Linear-Quadratic Control with Multiplicative Noise

arXiv 2510.02896 ↗First posted 3 Oct 2025eess.SY cs.AI cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Reinforcement Learning (RL) has emerged as a powerful framework for sequential decision-making in dynamic environments, particularly when system parameters are unknown. This paper investigates RL-based control for entropy-regularized linear-quadratic (LQ) control problems with multiplicative noise over an infinite time horizon. First, we adapt the regularized policy gradient (RPG) algorithm to stochastic optimal control settings, proving that despite the non-convexity of the problem, RPG converges globally under conditions of gradient domination and almost-smoothness. Second, based on zero-order optimization approach, we introduce a novel model free RL algorithm: Sample-based regularized policy gradient (SB-RPG). SB-RPG operates without knowledge of system parameters yet still retains strong theoretical guarantees of global convergence. Our model leverages entropy regularization to address the exploration versus exploitation trade-off inherent in RL. Numerical simulations validate the theoretical results and demonstrate the efficiency of SB-RPG in unknown-parameters environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 26 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

13 of 26 references.

References · 13

  1. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  2. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  3. 2018Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulatorcited by 250
  4. 2017A unified view of entropy-regularized Markov decision processescited by 182
  5. 2018Understanding the impact of entropy on policy optimizationcited by 176
  6. 2020Policy Gradient Methods for the Noisy Linear Quadratic Regulator over a Finite Horizoncited by 50
  7. 2022Convergence of policy gradient methods for finite-horizon exploratory linear-quadratic control problemscited by 25
  8. 2017Thompson Sampling for Linear-Quadratic Control Problemscited by 14
  9. 2017DeepChess: End-to-End Deep Neural Network for Automatic Learning in Chesscited by 14
  10. 2023Fast Policy Learning for Linear Quadratic Control with Entropy Regularizationcited by 13
  11. 2020Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Methodcited by 11
  12. 2020Model-free optimal control of discrete-time systems with additive and multiplicative noisescited by 8
1 more
  1. 2019Learning robust control for LQR systems with multiplicative noise via policy gradientcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Full error analysis of policy gradient learning algorithms for exploratory linear quadratic mean-field control problem in continuous time with common noise
  2. 2025Sample-Efficient Model-Free Policy Gradient Methods for Stochastic LQR via Robust Linear Regression
  3. 2025Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning
  4. 2024Convergence of Policy Gradient for Stochastic Linear-Quadratic Control Problem in Infinite Horizon
  5. 2025Convergence Guarantees of Model-free Policy Gradient Methods for LQR with Stochastic Data
  6. 2025Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problems
  7. 2026Reinforcement Learning in Real Option Models
  8. 2026Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models