# Uncertainty-based Modulation for Lifelong Learning

arXiv: https://arxiv.org/abs/2001.09822 · HTML: https://mlsanity.com/paper/2001.09822/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems](https://mlsanity.com/paper/2301.07799/) (2023)
- [Model-Free Generative Replay for Lifelong Reinforcement Learning: Application to Starcraft-2](https://mlsanity.com/paper/2208.05056/) (2022)
- [Incremental cluster validity index-guided online learning for performance and robustness to presentation order](https://mlsanity.com/paper/2108.07743/) (2021)
- [Improving the adaptive and continuous learning capabilities of artificial neural networks: Lessons from multi-neuromodulatory dynamics](https://mlsanity.com/paper/2501.06762/) (2025)
- [Bridging Neuroscience and AI: Environmental Enrichment as a Model for Forward Knowledge Transfer](https://mlsanity.com/paper/2405.07295/) (2024)

## Abstract (arXiv)

The creation of machine learning algorithms for intelligent agents capable of continuous, lifelong learning is a critical objective for algorithms being deployed on real-life systems in dynamic environments. Here we present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's ground-breaking Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of neuromodulatory mechanisms to enable continuous learning, including self-supervised and one-shot learning. Algorithm components were evaluated in a series of benchmark experiments that demonstrate stable learning without catastrophic forgetting. We also demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process. To this end, we integrated the algorithm into an embodied simulated drone agent. The experiments show that the algorithm is capable of continuous learning of new tasks and under changed conditions with high classification accuracy (greater than 94 percent) in a virtual environment, without catastrophic forgetting. The algorithm accepts high dimensional inputs from any state-of-the-art detection and feature extraction algorithms, making it a flexible addition to existing systems. We also describe future development efforts focused on imbuing the algorithm with mechanisms to seek out new knowledge as well as employ a broader range of neuromodulatory processes.