mlsanityData to 14 Oct 2026

Thermodynamics of Reinforcement Learning Curricula

arXiv 2603.12324 ↗First posted 12 Mar 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Connections between statistical mechanics and machine learning have repeatedly proven fruitful, providing insight into optimization, generalization, and representation learning. In this work, we follow this tradition by leveraging results from non-equilibrium thermodynamics to formalize curriculum learning in reinforcement learning (RL). In particular, we propose a geometric framework for RL by interpreting reward parameters as coordinates on a task manifold. We show that, by minimizing the excess thermodynamic work, optimal curricula correspond to geodesics in this task space. As an application of this framework, we provide an algorithm, "MEW" (Minimum Excess Work), to derive a principled schedule for temperature annealing in maximum-entropy RL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 17 of its 31 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

17 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025A Differential Perspective on Distributional Reinforcement Learning3 Jun
  2. 2025Average-Reward Soft Actor-Critic15 Jan

Other references · 15

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  4. 2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 482
  5. 2021Explaining Neural Scaling Lawscited by 363
  6. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  7. 2020Quantifying Differences in Reward Functionscited by 63
  8. 2018Fluctuation-dissipation relations for stochastic gradient descentcited by 48
  9. 2019The Value Function Polytope in Reinforcement Learningcited by 29
  10. 2022Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptationcited by 28
  11. 2022Dynamics-Aware Comparison of Learned Reward Functionscited by 17
  12. 2021Reinforcement learning of rare diffusive dynamicscited by 15
3 more
  1. 2024RVI-SAC: Average Reward Off-Policy Deep Reinforcement Learningcited by 6
  2. 2021Entropy Regularized Reinforcement Learning Using Large Deviation Theorycited by 5
  3. 2022Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviationscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning
  2. 2025Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning
  3. 2026Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization
  4. 2026Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
  5. 2025Causally Aligned Curriculum Learning
  6. 2024Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction
  7. 2026MePoly: Max Entropy Polynomial Policy Optimization
  8. 2025Maximizing Confidence Alone Improves Reasoning