mlsanityData to 14 Oct 2026

Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies

arXiv 2607.19928 ↗First posted 22 Jul 2026math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

We study entropy-regularized exploratory control in finite N-player stochastic differential games under a response model in which each player conditions on the opponents' currently realized actions and evaluates continuation with that profile frozen. The resulting Gibbs best responses form a system of full conditional densities, which need not admit a common joint law. We characterize joint realizability by a cross-partial condition on the entropy-scaled Hamiltonian gradients and, on simply connected action domains, by an equivalent entropy-weighted potential structure. When compatibility fails, a coordinate-path construction yields a joint density whose full conditionals satisfy explicit quadratic Kullback--Leibler bounds. We extend the analysis to stationary discounted problems and derive martingale and policy-improvement characterizations for learning the frozen response maps. A two-player linear-quadratic example illustrates the compatibility criterion and the associated learning procedure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 19 of its 59 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

19 of 59 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models27 Jun
  2. 2026Entropy Regularized Reinforcement Learning for Zero-Sum Stochastic Differential Games in a Regime-Switching Jump-Diffusion Process27 Jun
  3. 2025Continuous-time reinforcement learning for optimal switching over multiple regimes4 Dec
  4. 2024Sublinear Regret for a Class of Continuous-Time Linear-Quadratic Reinforcement Learning Problems24 Jul

Other references · 15

  1. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  2. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 101
  3. 2022q-Learning in Continuous Timecited by 91
  4. 2021Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approachcited by 84
  5. 2020Entropy Regularization for Mean Field Games with Learningcited by 58
  6. 2023Continuous-time q-learning for mean-field control problemscited by 22
  7. 2020Exploratory LQG Mean Field Games with Entropy Regularizationcited by 15
  8. 2024Unified continuous-time q-learning for mean-field game and mean-field control problemscited by 13
  9. 2024Regret of exploratory policy improvement and q-learningcited by 12
  10. 2026Continuous-time q-learning for mean-field control with common noise, part-I: Theoretical foundationscited by 6
  11. 2019Entropy-Regularized Stochastic Gamescited by 5
  12. 2022Square-root regret bounds for continuous-time episodic Markov decision processescited by 5
3 more
  1. 2025Data-Driven Exploration for a Class of Continuous-Time Indefinite Linear--Quadratic Reinforcement Learning Problemscited by 3
  2. 2026Learning Distributed Equilibria in Linear-Quadratic Stochastic Differential Games: An α-Potential Approachcited by 2
  3. 2024A Coupled Optimization Framework for Correlated Equilibria in Normal-Form Gamecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
  2. 2026Scalable Maximum Entropy Reinforcement Learning for Diffusion Policies via Adjoint Matching
  3. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
  4. 2026Entropy Regularization Improves Policy Robustness in Continuous-Time Reinforcement Learning
  5. 2026A Comparative Theoretical Analysis of Entropy Control Methods in Reinforcement Learning
  6. 2026Reinforcement Learning in Real Option Models
  7. 2025Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning
  8. 2026Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning