mlsanityData to 7 Oct 2026

Evidence on the Regularisation Properties of Maximum-Entropy Reinforcement Learning

arXiv 2501.17115 ↗First posted 28 Jan 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The generalisation and robustness properties of policies learnt through Maximum-Entropy Reinforcement Learning are investigated on chaotic dynamical systems with Gaussian noise on the observable. First, the robustness under noise contamination of the agent's observation of entropy regularised policies is observed. Second, notions of statistical learning theory, such as complexity measures on the learnt model, are borrowed to explain and predict the phenomenon. Results show the existence of a relationship between entropy-regularised policy optimisation and robustness to noise, which can be described by the chosen complexity measures.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

25 of 48 references.

References · 25

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  5. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  6. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  7. 2017Deep Reinforcement Learning that Matterscited by 727
  8. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  9. 2017Exploring Generalization in Deep Learningcited by 595
  10. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  11. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  12. 2015Norm-Based Capacity Control in Neural Networkscited by 299
13 more
  1. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  2. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  3. 2017A unified view of entropy-regularized Markov decision processescited by 174
  4. 2018Understanding the impact of entropy on policy optimizationcited by 171
  5. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  6. 2016Eigenvalues of the Hessian in Deep Learning: Singularity and Beyondcited by 156
  7. 2022Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learningcited by 122
  8. 2018Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approachcited by 89
  9. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
  10. 2020Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalizationcited by 61
  11. 2018How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experimentscited by 56
  12. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  13. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Maximum Likelihood Reinforcement Learningcited by 29
  2. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscapecited by 3
  3. 2026HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systemscited by 0