mlsanityData to 14 Oct 2026

Ergodic Risk Measures: Towards a Risk-Aware Foundation for Continual Reinforcement Learning

arXiv 2510.02945 ↗First posted 3 Oct 2025cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Continual reinforcement learning (continual RL) seeks to formalize the notions of lifelong learning and endless adaptation in RL. In particular, the aim of continual RL is to develop RL agents that can maintain a careful balance between retaining useful information and adapting to new situations. To date, continual RL has been explored almost exclusively through the lens of risk-neutral decision-making, in which the agent aims to optimize the expected long-run performance. In this work, we present the first formal theoretical treatment of continual RL through the lens of risk-aware decision-making, in which the behaviour of the agent is directed towards optimizing a measure of long-run performance beyond the mean. In particular, we show that the classical theory of risk measures, widely used as a theoretical foundation in non-continual risk-aware RL, is, in its current form, incompatible with continual learning. Then, building on this insight, we extend risk measure theory into the continual setting by introducing a new class of ergodic risk measures, and showing that it is compatible with continual learning. Finally, we provide a case study of continual risk-aware learning, along with empirical results, which show the intuitive appeal of ergodic risk measures in continual settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 15 of its 32 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

15 of 32 references.

References · 15

  1. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  2. 2023Loss of Plasticity in Continual Deep Reinforcement Learningcited by 121
  3. 2023A Definition of Continual Reinforcement Learningcited by 101
  4. 2023Continual Learning as Computationally Constrained Reinforcement Learningcited by 37
  5. 2021Autonomous Reinforcement Learning: Formalism and Benchmarkingcited by 33
  6. 2022Settling the Reward Hypothesiscited by 33
  7. 2021Formalizing the Generalization-Forgetting Trade-off in Continual Learningcited by 24
  8. 2021Risk-Aware Transfer in Reinforcement Learning using Successor Featurescited by 12
  9. 2025Plasticity as the Mirror of Empowermentcited by 12
  10. 2022Meta-Gradients in Non-Stationary Environmentscited by 11
  11. 2025Return Capping: Sample-Efficient CVaR Policy Gradient Optimisationcited by 6
  12. 2025A Differential Perspective on Distributional Reinforcement Learningcited by 5
3 more
  1. 2022Risk-Sensitive Markov Decision Processes with Long-Run CVaR Criterioncited by 4
  2. 2024Burning RED: Unlocking Subtask-Driven Reinforcement Learning and Risk-Awareness in Average-Reward Markov Decision Processescited by 2
  3. 2025Planning and Learning in Average Risk-aware MDPscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking the Foundations for Continual Reinforcement Learning
  2. 2025A Survey of Continual Reinforcement Learning
  3. 2025Advancements and Challenges in Continual Reinforcement Learning: A Comprehensive Review
  4. 2026Safe Continual Reinforcement Learning in Non-stationary Environments
  5. 2025Continual Reinforcement Learning for Cyber-Physical Systems: Lessons Learned and Open Challenges
  6. 2025The Cell Must Go On: Agar.io for Continual Reinforcement Learning
  7. 2026Position: Deployed Reinforcement Learning should be Continual
  8. 2025On the Design of Safe Continual RL Methods for Control of Nonlinear Systems