# Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models

arXiv: https://arxiv.org/abs/2606.28671 · HTML: https://mlsanity.com/paper/2606.28671/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 27 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 12 of 12)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithms](https://mlsanity.com/paper/2111.11232/) (2021)
- [Maximum Entropy-Regularized Multi-Goal Reinforcement Learning](https://mlsanity.com/paper/1905.08786/) (2019)
- [Exploratory HJB equations and their convergence](https://mlsanity.com/paper/2109.10269/) (2021)
- [Actor-Critic Method for High Dimensional Static Hamilton--Jacobi--Bellman Partial Differential Equations based on Neural Networks](https://mlsanity.com/paper/2102.11379/) (2021)
- [Exploration versus exploitation in reinforcement learning: a stochastic control approach](https://mlsanity.com/paper/1812.01552/) (2018)
- [Continuous-time q-Learning for Jump-Diffusion Models under Tsallis Entropy](https://mlsanity.com/paper/2407.03888/) (2024)
- [Effective Multi-Agent Deep Reinforcement Learning Control with Relative Entropy Regularization](https://mlsanity.com/paper/2309.14727/) (2023)
- [Zero-Sum Stackelberg Stochastic Linear-Quadratic Differential Games](https://mlsanity.com/paper/2109.14893/) (2021)
- [Linear quadratic leader-follower stochastic differential games for mean-field switching diffusions](https://mlsanity.com/paper/2208.00347/) (2022)
- [Closed-Loop Solvability of Linear Quadratic Mean-Field Type Stackelberg Stochastic Differential Games](https://mlsanity.com/paper/2303.07544/) (2023)
- [Exploratory Utility Maximization Problem with Tsallis Entropy](https://mlsanity.com/paper/2502.01269/) (2025)

## Built on by (most cited first, 1 of 1)

- [Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies](https://mlsanity.com/paper/2607.19928/) (2026)

## Abstract (arXiv)

Stackelberg differential games (SDGs) provide a powerful framework for hierarchical decision-making in stochastic and continuous-time environments, yet their solution remains computationally challenging due to the complexity of traditional dynamic programming and Hamilton-Jacobi-Bellman-Isaacs (HJBI) methods, especially in high-dimensional systems. This paper proposes an entropy-regularized reinforcement learning (ERRL) approach for linear-quadratic SDGs (LQ-SDGs) within a continuous-time diffusion framework governed by Markovian regime switching. The key innovation lies in deriving exploratory weakly-coupled HJBI equations with entropy regularization, which promotes stochastic policies that actively avoid suboptimal equilibria -- a limitation of classical SDG methods. Neural networks are integrated to approximate regime-dependent value functions and solve high-dimensional partial differential equations (PDEs) efficiently, while a novel sampling technique enhances computational tractability. Numerical results demonstrate the effectiveness of the framework compared to conventional approaches, particularly in escaping suboptimal traps through exploratory policies. The study highlights the critical role of entropy regularization and neural network approximations in achieving robust solutions for hierarchical decision-making problems under abrupt environmental shifts.