Sharp Spectral Thresholds for Logit Fixed Points
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Softmax feedback systems are a common mathematical core of entropy-regularized reinforcement learning, logit game dynamics, population choice, and mean-field variational updates. Their central stability question is simple: when does a self-reinforcing softmax system produce a unique and globally predictable outcome? Classical theory gives a conservative answer. By treating softmax as a unit-scale response, it certifies stability only in a strongly randomized regime. We prove that the classical approach misses an entire stable regime and does not identify the point at which the qualitative change truly occurs. For finite-dimensional affine logit systems, the sharp dimension-free Euclidean threshold is β|Pi WPi|mathcal T→mathcal T<2, rather than the previously used condition, which certifies stability only while the softmax system remains safely over-regularized. Our theorem fills the previously missing pre-bifurcation regime, extending stability guarantees for affine softmax feedback systems to reward-responsive yet globally predictable systems. It enlarges the certified stability boundary for these systems and identifies where the model genuinely undergoes a phase transition.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 9 of its 48 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
9 of 48 references.
References · 9
- 2019A Theory of Regularized Markov Decision Processescited by 286
- 2017A unified view of entropy-regularized Markov decision processescited by 182
- 2020Fast Global Convergence of Natural Policy Gradient Methods with Entropy Regularizationcited by 176
- 2017On the Properties of the Softmax Function with Application in Game Theory and Reinforcement Learningcited by 142
- 2021Beyond Exact Gradients: Convergence of Stochastic Soft-Max Policy Gradient Methods with Entropy Regularizationcited by 25
- 2025Softmax is 1/2-Lipschitz: A tight bound across all ℓp normscited by 22
- 2017Positive feedback in coordination games: stochastic evolutionary dynamics and the logit choice rulecited by 1
- 2023On the stability of the logit dynamics in population gamescited by 1
- 2024An Analysis of Logit Learning with the r-Lambert Functioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Logit Dynamics in Softmax Policy Gradient Methods
- 2024Finite-Time Error Analysis of Soft Q-Learning: Switching System Approach
- 2026Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning
- 2026Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs
- 2026On the Convergence of Semi-Gradient TD: A Mean Field Perspective
- 2026Rationalizing Boltzmann Rationality: An Axiomatic Characterization of Entropy-Regularized Policies
- 2026Soft Deterministic Policy Gradient with Gaussian Smoothing
- 2025Soft Fitted Q-Iteration without Bellman Completeness: Occupancy Reweighting and Temperature Annealing