# Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity

arXiv: https://arxiv.org/abs/2002.10585 · HTML: https://mlsanity.com/paper/2002.10585/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2020.

## Built on by (most cited first, 30 of 41)

- [Linear Transformers Are Secretly Fast Weight Programmers](https://mlsanity.com/paper/2102.11174/) (2021)
- [MLGym: A New Framework and Benchmark for Advancing AI Research Agents](https://mlsanity.com/paper/2502.14499/) (2025)
- [Going Beyond Linear Transformers with Recurrent Fast Weight Programmers](https://mlsanity.com/paper/2106.06295/) (2021)
- [A Modern Self-Referential Weight Matrix That Learns to Modify Itself](https://mlsanity.com/paper/2202.05780/) (2022)
- [Brain-inspired learning in artificial neural networks: a review](https://mlsanity.com/paper/2305.11252/) (2023)
- [Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways](https://mlsanity.com/paper/2210.16414/) (2022)
- [Adaptive Convolutions with Per-pixel Dynamic Filter Atom](https://mlsanity.com/paper/2108.07895/) (2021)
- [Context Meta-Reinforcement Learning via Neuromodulation](https://mlsanity.com/paper/2111.00134/) (2021)
- [Meta-Learning Bidirectional Update Rules](https://mlsanity.com/paper/2104.04657/) (2021)
- [Fast weight programming and linear transformers: from machine learning to neurobiology](https://mlsanity.com/paper/2508.08435/) (2025)
- [Metalearning Continual Learning Algorithms](https://mlsanity.com/paper/2312.00276/) (2023)
- [Short-Term Plasticity Neurons Learning to Learn and Forget](https://mlsanity.com/paper/2206.14048/) (2022)
- [Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning](https://mlsanity.com/paper/2109.03554/) (2021)
- [SpikePropamine: Differentiable Plasticity in Spiking Neural Networks](https://mlsanity.com/paper/2106.02681/) (2021)
- [Learning to acquire novel cognitive tasks with evolution, plasticity and meta-meta-learning](https://mlsanity.com/paper/2112.08588/) (2021)
- [Meta-Reinforcement Learning with Self-Modifying Networks](https://mlsanity.com/paper/2202.02363/) (2022)
- [Hebbian and Gradient-based Plasticity Enables Robust Memory and Rapid Learning in RNNs](https://mlsanity.com/paper/2302.03235/) (2023)
- [Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learning](https://mlsanity.com/paper/2204.04297/) (2022)
- [Temporal-Difference Learning Using Distributed Error Signals](https://mlsanity.com/paper/2411.03604/) (2024)
- [Learning to learn online with neuromodulated synaptic plasticity in spiking neural networks](https://mlsanity.com/paper/2206.12520/) (2022)
- [Biological connectomes as a representation for the architecture of artificial neural networks](https://mlsanity.com/paper/2209.14406/) (2022)
- [Improving Performance in Continual Learning Tasks using Bio-Inspired Architectures](https://mlsanity.com/paper/2308.04539/) (2023)
- [Look-Ahead Selective Plasticity for Continual Learning of Visual Tasks](https://mlsanity.com/paper/2311.01617/) (2023)
- [Improving the adaptive and continuous learning capabilities of artificial neural networks: Lessons from multi-neuromodulatory dynamics](https://mlsanity.com/paper/2501.06762/) (2025)
- [Same World, Differently Given: History-Dependent Perceptual Reorganization in Artificial Agents](https://mlsanity.com/paper/2604.04637/) (2026)
- [A Modulation Layer to Increase Neural Network Robustness Against Data Quality Issues](https://mlsanity.com/paper/2107.08574/) (2021)
- [Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks](https://mlsanity.com/paper/2211.09440/) (2022)
- [NM-Hebb: Coupling Local Hebbian Plasticity with Metric Learning for More Accurate and Interpretable CNNs](https://mlsanity.com/paper/2508.19896/) (2025)
- [Enabling Robust In-Context Memory and Rapid Task Adaptation in Transformers with Hebbian and Gradient-Based Plasticity](https://mlsanity.com/paper/2510.21908/) (2025)
- [RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers](https://mlsanity.com/paper/2601.00426/) (2026)

## Abstract (arXiv)

The impressive lifelong learning in animal brains is primarily enabled by plastic changes in synaptic connectivity. Importantly, these changes are not passive, but are actively controlled by neuromodulation, which is itself under the control of the brain. The resulting self-modifying abilities of the brain play an important role in learning and adaptation, and are a major basis for biological reinforcement learning. Here we show for the first time that artificial neural networks with such neuromodulated plasticity can be trained with gradient descent. Extending previous work on differentiable Hebbian plasticity, we propose a differentiable formulation for the neuromodulation of plasticity. We show that neuromodulated plasticity improves the performance of neural networks on both reinforcement learning and supervised learning tasks. In one task, neuromodulated plastic LSTMs with millions of parameters outperform standard LSTMs on a benchmark language modeling task (controlling for the number of parameters). We conclude that differentiable neuromodulation of plasticity offers a powerful new framework for training neural networks.