Lifelong Reinforcement Learning via Neuromodulation
Not on the map. Why?
Abstract (arXiv)
Navigating multiple tasksunicodex2014for instance in succession as in continual or lifelong learning, or in distributions as in meta or multi-task learningunicodex2014requires some notion of adaptation. Evolution over timescales of millennia has imbued humans and other animals with highly effective adaptive learning and decision-making strategies. Central to these functions are so-called neuromodulatory systems. In this work we introduce an abstract framework for integrating theories and evidence from neuroscience and the cognitive sciences into the design of adaptive artificial reinforcement learning algorithms. We give a concrete instance of this framework built on literature surrounding the neuromodulators Acetylcholine (ACh) and Noradrenaline (NA), and empirically validate the effectiveness of the resulting adaptive algorithm in a non-stationary multi-armed bandit problem. We conclude with a theory-based experiment proposal providing an avenue to link our framework back to efforts in experimental neuroscience.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 129 references that are in the map. First posted on arXiv 15 Aug 2024.
Builds on
27 of 129 references.
References · 27
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
- 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 903
- 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
- 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
- 2017A Distributional Perspective on Reinforcement Learningcited by 743
- 2016Deep Exploration via Bootstrapped DQNcited by 518
- 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
- 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
15 more
- 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
- 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
- 2018Meta-Gradient Reinforcement Learningcited by 156
- 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
- 2021DEUP: Direct Epistemic Uncertainty Predictioncited by 102
- 2020Chip Placement with Deep Reinforcement Learningcited by 83
- 2016Towards an integration of deep learning and neurosciencecited by 77
- 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
- 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
- 2020Deep Reinforcement Learning and its Neuroscientific Implicationscited by 33
- 2018Continual Reinforcement Learning with Complex Synapsescited by 30
- 2022Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learningcited by 27
- 2020Learning Adaptive Exploration Strategies in Dynamic Environments Through Informed Policy Regularizationcited by 15
- 2014Kalman Temporal Differencescited by 10
- 2020Geometric framework to predict structure from function in neural networkscited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.