# Do Artificial Neural Networks Need Dendritic Nanotubes? Two Reasons for Shrinking Importance Estimates in Continual Learning

arXiv: https://arxiv.org/abs/2610.14258 · HTML: https://mlsanity.com/paper/2610.14258/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interference](https://mlsanity.com/paper/1810.11910/) (2018)
- [Progress & Compress: A scalable framework for continual learning](https://mlsanity.com/paper/1805.06370/) (2018)
- [New insights and perspectives on the natural gradient method](https://mlsanity.com/paper/1412.1193/) (2014)
- [Variational Continual Learning](https://mlsanity.com/paper/1710.10628/) (2017)
- [Limitations of the Empirical Fisher Approximation for Natural Gradient Descent](https://mlsanity.com/paper/1905.12558/) (2019)
- [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1805.07810/) (2018)
- [Uncertainty-based Continual Learning with Adaptive Regularization](https://mlsanity.com/paper/1905.11614/) (2019)
- [Continual Learning with Node-Importance based Adaptive Group Sparse Regularization](https://mlsanity.com/paper/2003.13726/) (2020)
- [Uncertainty-guided Continual Learning with Bayesian Neural Networks](https://mlsanity.com/paper/1906.02425/) (2019)
- [On Quadratic Penalties in Elastic Weight Consolidation](https://mlsanity.com/paper/1712.03847/) (2017)
- [Continual Reinforcement Learning with Complex Synapses](https://mlsanity.com/paper/1802.07239/) (2018)
- [Synaptic metaplasticity in binarized neural networks](https://mlsanity.com/paper/2101.07592/) (2021)
- [On the Computation of the Fisher Information in Continual Learning](https://mlsanity.com/paper/2502.11756/) (2025)
- [Bayesian continual learning and forgetting in neural networks](https://mlsanity.com/paper/2504.13569/) (2025)
- [Affect and Effect: Limitations of regularisation-based continual learning in EEG-based emotion classification](https://mlsanity.com/paper/2601.07858/) (2026)

## Abstract (arXiv)

Regularization-based continual learning protects each parameter in proportion to an estimate of its importance, yet these estimates come from finite samples and heuristics and are used as if they were exact. We report four studies that start from a biologically inspired mechanism and end with a statistical explanation of it. Motivated by dendritic nanotubes, we let transient lateral links diffuse importance tags between hidden units. Links between co-active units gave no significant gain, but random links with lifetimes of 25 or 200 steps raised final accuracy on 20-task Permuted MNIST by 1.5 to 2.6 points. We show that random diffusion implements linear shrinkage of importance toward the layer mean in expectation. A simulation of the link process alone predicts shrinkage weights of 0.23 and 0.52 for these lifetimes, and graph-free shrinkage at 0.25 and 0.5 reproduces both the accuracy and the dispersion of the consolidated importance, while every activity-dependent rule did worse than random links. A five-seed screen across EWC, SI, and MAS found nominal gains for SI and MAS on Permuted MNIST and losses on task-incremental Split CIFAR-100, where forgetting was small and shrinkage cost plasticity. Varying the number of importance samples then separates two mechanisms on Permuted MNIST. For EWC, the gain fell by about ten points per tenfold increase in samples, from 19.6 points with 10 samples to -0.4 with 1000, a variance effect. For MAS, a gain of about one point persisted at 1000 samples, consistent with a bias in the importance measure rather than noise. Shrinkage had no detectable effect for MAS on a domain-incremental CIFAR-100 benchmark. Finally, we show that a standard heteroscedastic empirical Bayes rule shrinks the largest Fisher-type estimates most, which may explain its poor results, and point to a log-scale alternative.