mlsanityData to 14 Oct 2026

Model-Free Risk-Sensitive Reinforcement Learning

arXiv 2111.02907 ↗First posted 4 Nov 2021cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We extend temporal-difference (TD) learning in order to obtain risk-sensitive, model-free reinforcement learning algorithms. This extension can be regarded as modification of the Rescorla-Wagner rule, where the (sigmoidal) stimulus is taken to be either the event of over- or underestimating the TD target. As a result, one obtains a stochastic approximation rule for estimating the free energy from i.i.d. samples generated by a Gaussian distribution with unknown mean and variance. Since the Gaussian free energy is known to be a certainty-equivalent sensitive to the mean and the variance, the learning rule has applications in risk-sensitive decision-making.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 12 of its 42 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

12 of 42 references.

References · 12

  1. 2016Concrete Problems in AI Safetycited by 1,642
  2. 2016Learning to reinforcement learncited by 377
  3. 2013Risk-sensitive Reinforcement Learningcited by 130
  4. 2017AI Safety Gridworldscited by 110
  5. 2016Research Priorities for Robust and Beneficial Artificial Intelligencecited by 74
  6. 2019Meta-learning of Sequential Strategiescited by 70
  7. 2020Meta-trained agents implement Bayes-optimal agentscited by 38
  8. 2021Regularized Policies are Reward Robustcited by 32
  9. 2019If MaxEnt RL is the Answer, What is the Question?cited by 27
  10. 2014Exploration vs Exploitation vs Safety: Risk-averse Multi-Armed Banditscited by 26
  11. 2016Planning with Information-Processing Constraints and Model Uncertainty in Markov Decision Processescited by 7
  12. 2014An Adversarial Interpretation of Information-Theoretic Bounded Rationalitycited by 6

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 0; 2025: 2.

7 papers.

  1. 2023Single-Trajectory Distributionally Robust Reinforcement Learningcited by 16
  2. 2022Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shapingcited by 6
  3. 2023Learning Diverse Risk Preferences in Population-based Self-playcited by 6
  4. 2025Beyond CVaR: Leveraging Static Spectral Risk Measures for Enhanced Decision-Making in Distributional Reinforcement Learningcited by 6
  5. 2022Beyond Bayes-optimality: meta-learning what you know you don't knowcited by 1
  6. 2023EduGym: An Environment and Notebook Suite for Reinforcement Learning Educationcited by 0
  7. 2025Risk-Averse Reinforcement Learning with Itakura-Saito Losscited by 0