mlsanityData to 7 Oct 2026

Maximum diffusion reinforcement learning

arXiv 2309.15293 ↗First posted 26 Sep 2023cs.LG cond-mat.stat-mech cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Robots and animals both experience the world through their bodies and senses. Their embodiment constrains their experiences, ensuring they unfold continuously in space and time. As a result, the experiences of embodied agents are intrinsically correlated. Correlations create fundamental challenges for machine learning, as most techniques rely on the assumption that data are independent and identically distributed. In reinforcement learning, where data are directly collected from an agent's sequential experiences, violations of this assumption are often unavoidable. Here, we derive a method that overcomes this issue by exploiting the statistical mechanics of ergodic processes, which we term maximum diffusion reinforcement learning. By decorrelating agent experiences, our approach provably enables single-shot learning in continuous deployments over the course of individual task attempts. Moreover, we prove our approach generalizes well-known maximum entropy techniques, and robustly exceeds state-of-the-art performance across popular benchmarks. Our results at the nexus of physics, learning, and control form a foundation for transparent and reliable decision-making in embodied reinforcement learning agents.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 41 of its 113 references that are in the map. First posted on arXiv 26 Sep 2023.

Builds on

41 of 113 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  10. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 299
  11. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  12. 2021Reinforcement learning based recommender systems: A surveycited by 187
29 more
  1. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  2. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 181
  3. 2020Revisiting Fundamentals of Experience Replaycited by 171
  4. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  5. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  6. 2017Parameter Space Noise for Explorationcited by 158
  7. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  8. 2017On the Sample Complexity of the Linear Quadratic Regulatorcited by 151
  9. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  10. 2017A Deeper Look at Experience Replaycited by 104
  11. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  12. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 92
  13. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  14. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 66
  15. 2021Beyond No Regret: Instance-Dependent PAC Reinforcement Learningcited by 41
  16. 2021Active Learning in Robotics: A Review of Control Principlescited by 26
  17. 2022You Only Live Once: Single-Life Reinforcement Learningcited by 23
  18. 2021Linear Systems can be Hard to Learncited by 20
  19. 2014On Submodularity and Controllability in Complex Dynamical Networkscited by 10
  20. 2022Learning to Control Linear Systems can be Hardcited by 10
  21. 2013Predictive PAC Learning and Process Decompositionscited by 9
  22. 2017Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3cited by 9
  23. 2017Ergodic Exploration of Distributed Informationcited by 9
  24. 2017Real-Time Area Coverage and Target Localization using Receding-Horizon Ergodic Explorationcited by 7
  25. 2021Maximum Entropy Differential Dynamic Programmingcited by 5
  26. 2014Submodularity of Energy Related Controllability Metricscited by 2
  27. 2016Noise Response Data Reveal Novel Controllability Gramian for Nonlinear Network Dynamicscited by 2
  28. 2017Sequential Action Control: Closed-Form Optimal Control for Nonlinear and Nonsmooth Systemscited by 2
  29. 2021Low rattling: A predictive principle for self-organization in active collectivescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2025Efficient Reinforcement Learning with Semantic and Token Entropy for LLM Reasoningcited by 6
  2. 2024FairDiffusion: Enhancing Equity in Latent Diffusion Models via Fair Bayesian Perturbationcited by 4
  3. 2025Distributionally Robust Free Energy Principle for Decision-Makingcited by 4
  4. 2024Preferred-Action-Optimized Diffusion Policies for Offline Reinforcement Learningcited by 3
  5. 2025Volumetric Ergodic Controlcited by 1
  6. 2026Physics-informed offline reinforcement learning eliminates catastrophic fuel waste in maritime routingcited by 1
  7. 2026Damage Adaptation in Seconds for Architected Materialscited by 0
  8. 2026The Open Ant: A Robot Platform for Reinforcement Learning Researchcited by 0