mlsanityData to 7 Oct 2026

Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models

arXiv 2606.28671 ↗First posted 27 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Stackelberg differential games (SDGs) provide a powerful framework for hierarchical decision-making in stochastic and continuous-time environments, yet their solution remains computationally challenging due to the complexity of traditional dynamic programming and Hamilton-Jacobi-Bellman-Isaacs (HJBI) methods, especially in high-dimensional systems. This paper proposes an entropy-regularized reinforcement learning (ERRL) approach for linear-quadratic SDGs (LQ-SDGs) within a continuous-time diffusion framework governed by Markovian regime switching. The key innovation lies in deriving exploratory weakly-coupled HJBI equations with entropy regularization, which promotes stochastic policies that actively avoid suboptimal equilibria -- a limitation of classical SDG methods. Neural networks are integrated to approximate regime-dependent value functions and solve high-dimensional partial differential equations (PDEs) efficiently, while a novel sampling technique enhances computational tractability. Numerical results demonstrate the effectiveness of the framework compared to conventional approaches, particularly in escaping suboptimal traps through exploratory policies. The study highlights the critical role of entropy regularization and neural network approximations in achieving robust solutions for hierarchical decision-making problems under abrupt environmental shifts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 34 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

12 of 34 references.

References · 12

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 76
  3. 2019Maximum Entropy-Regularized Multi-Goal Reinforcement Learningcited by 39
  4. 2021Exploratory HJB equations and their convergencecited by 28
  5. 2021Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networkscited by 27
  6. 2018Exploration versus exploitation in reinforcement learning: a stochastic control approachcited by 17
  7. 2024Continuous-time q-Learning for Jump-Diffusion Models under Tsallis Entropycited by 7
  8. 2021Zero-Sum Stackelberg Stochastic Linear-Quadratic Differential Gamescited by 3
  9. 2023Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularizationcited by 3
  10. 2022Linear quadratic leader-follower stochastic differential games for mean-field switching diffusionscited by 2
  11. 2023Closed-Loop Solvability of Linear Quadratic Mean-Field Type Stackelberg Stochastic Differential Gamescited by 1
  12. 2025Exploratory Utility Maximization Problem with Tsallis Entropycited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning
  2. 2025Policy Transfer for Continuous-Time Reinforcement Learning: A (Rough) Differential Equation Approach
  3. 2026A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning
  4. 2025DIME:Diffusion-Based Maximum Entropy Reinforcement Learning
  5. 2025Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
  6. 2026Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching
  7. 2024Maximum Causal Entropy IRL in Mean-Field Games and GNEP Framework for Forward RL
  8. 2025EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learning