Improving the adaptive and continuous learning capabilities of artificial neural networks: Lessons from multi-neuromodulatory dynamics
Not on the map. Why?
Abstract (arXiv)
Continuous adaptive learning, the ability to adapt to the environment and keep improving performance, is a hallmark of natural intelligence. Biological organisms excel in acquiring, transferring, and retaining knowledge while adapting to volatile environments, making them a source of inspiration for artificial neural networks (ANNs). This study explores how neuromodulation, a building block of learning in biological systems, can help address catastrophic forgetting and enhance the robustness of ANNs in continual learning. Driven by neuromodulators including dopamine (DA), acetylcholine (ACh), serotonin (5-HT) and noradrenaline (NA), neuromodulatory processes in the brain operate at multiple scales, facilitating dynamic responses to environmental changes through mechanisms ranging from local synaptic plasticity to global network-wide adaptability. Importantly, the relationship between neuromodulators and their interplay in modulating sensory and cognitive processes is more complex than previously expected, demonstrating a "many-to-many" neuromodulator-to-task mapping. To inspire neuromodulation-aware learning rules, we highlight (i) how multi-neuromodulatory interactions enrich single-neuromodulator-driven learning, (ii) the impact of neuromodulators across multiple spatio-temporal scales, and correspondingly, (iii) strategies for approximating and integrating neuromodulated learning processes in ANNs, and (iv) an architectural-general formulation of multi-neuromodulatory dynamics. We also present a conceptual study to showcase how neuromodulation-inspired mechanisms, such as DA-driven reward processing and NA-based cognitive flexibility, can enhance ANN performance in a Go/No-Go task. Though multi-scale neuromodulation, we aim to bridge the gap between biological and artificial learning, paving the way for ANNs with greater flexibility, robustness, and adaptability.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 67 of its 384 references that are in the map. First posted on arXiv 12 Jan 2025.
Builds on
67 of 384 references.
References · 67
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2016Progressive Neural Networkscited by 1,361
- 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
- 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,329
- 2019A Comprehensive Survey on Transfer Learningcited by 1,160
38 more
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2018Experience Replay for Continual Learningcited by 1,095
- 2016HyperNetworkscited by 1,027
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
- 2017A Survey on Multi-Task Learningcited by 840
- 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
- 2019GNNExplainer: Generating Explanations for Graph Neural Networkscited by 737
- 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
- 2016Learning to learn by gradient descent by gradient descentcited by 620
- 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
- 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 259
- 2018Online Learning: A Comprehensive Surveycited by 205
- 2016Grad-CAM: Why did you say that?cited by 161
- 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 97
- 2019Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNscited by 90
- 2023SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networkscited by 88
- 2020Learning to Continually Learncited by 84
- 2021Relating transformers to models and neural representations of the hippocampal formationcited by 62
- 2020Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticitycited by 41
- 2021Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environmentscited by 25
- 2023Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptationcited by 23
- 2022Single-phase deep learning in cortico-cortical networkscited by 19
- 2020Online Few-shot Gesture Learning on a Neuromorphic Processorcited by 18
- 2016Multi-scale brain networkscited by 16
- 2018Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviourscited by 16
- 2021Drawing Inspiration from Biological Dendrites to Empower Artificial Neural Networkscited by 15
- 2022Neuromorphic Artificial Intelligence Systemscited by 12
- 2022Continual task learning in natural and artificial agentscited by 12
- 2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
- 2019Biologically inspired sleep algorithm for artificial neural networkscited by 10
- 2019A Review on Neural Network Models of Schizophrenia and Autism Spectrum Disordercited by 8
- 2023The Quest for an Integrated Set of Neural Mechanisms Underlying Object Recognition in Primatescited by 8
- 2024Transformers and Cortical Waves: Encoders for Pulling In Context Across Timecited by 7
- 2021On out-of-distribution detection with Bayesian neural networkscited by 6
- 2023Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networkscited by 6
- 2023Leveraging dendritic properties to advance machine learning and neuro-inspired computingcited by 6
- 2020Uncertainty-based Modulation for Lifelong Learningcited by 5
50 most cited of 67 All 67 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.