# Neurogenesis Deep Learning

arXiv: https://arxiv.org/abs/1612.03770 · HTML: https://mlsanity.com/paper/1612.03770/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2016.

## Built on by (most cited first, 30 of 32)

- [Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models](https://mlsanity.com/paper/2305.10120/) (2023)
- [DEMix Layers: Disentangling Domains for Modular Language Modeling](https://mlsanity.com/paper/2108.05036/) (2021)
- [Continual Learning and Catastrophic Forgetting](https://mlsanity.com/paper/2403.05175/) (2024)
- [Replay in Deep Learning: Current Approaches and Missing Biological Elements](https://mlsanity.com/paper/2104.04132/) (2021)
- [Learngene: From Open-World to Your Learning Task](https://mlsanity.com/paper/2106.06788/) (2021)
- [SIESTA: Efficient Online Continual Learning with Sleep](https://mlsanity.com/paper/2303.10725/) (2023)
- [Maslow's Hammer for Catastrophic Forgetting: Node Re-Use vs Node Activation](https://mlsanity.com/paper/2205.09029/) (2022)
- [Self-Expanding Neural Networks](https://mlsanity.com/paper/2307.04526/) (2023)
- [Optimal Protocols for Continual Learning via Statistical Physics and Control Theory](https://mlsanity.com/paper/2409.18061/) (2024)
- [Overcoming the Stability Gap in Continual Learning](https://mlsanity.com/paper/2306.01904/) (2023)
- [Self-Activating Neural Ensembles for Continual Reinforcement Learning](https://mlsanity.com/paper/2301.00141/) (2022)
- [FFNB: Forgetting-Free Neural Blocks for Deep Continual Visual Learning](https://mlsanity.com/paper/2111.11366/) (2021)
- [Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience](https://mlsanity.com/paper/2112.14146/) (2021)
- [Addressing Client Drift in Federated Continual Learning with Adaptive Optimization](https://mlsanity.com/paper/2203.13321/) (2022)
- [Incremental Deep Neural Network Learning using Classification Confidence Thresholding](https://mlsanity.com/paper/2106.11437/) (2021)
- [Augmentative Topology Agents For Open-Ended Learning](https://mlsanity.com/paper/2210.11442/) (2022)
- [Cold Start Streaming Learning for Deep Networks](https://mlsanity.com/paper/2211.04624/) (2022)
- [Towards Heterogeneous Clients with Elastic Federated Learning](https://mlsanity.com/paper/2106.09433/) (2021)
- [Lifelong Learning for Neural powered Mixed Integer Programming](https://mlsanity.com/paper/2208.12226/) (2022)
- [Incremental Class Learning using Variational Autoencoders with Similarity Learning](https://mlsanity.com/paper/2110.01303/) (2021)
- [Accretionary Learning with Deep Neural Networks](https://mlsanity.com/paper/2111.10857/) (2021)
- [Reviewing continual learning from the perspective of human-level intelligence](https://mlsanity.com/paper/2111.11964/) (2021)
- [Object Pursuit: Building a Space of Objects via Discriminative Weight Generation](https://mlsanity.com/paper/2112.07954/) (2021)
- [Learning Citywide Patterns of Life from Trajectory Monitoring](https://mlsanity.com/paper/2206.15352/) (2022)
- [Continual Few-Shot Learning with Adversarial Class Storage](https://mlsanity.com/paper/2207.12303/) (2022)
- [Review learning: Real world validation of privacy preserving continual learning across medical institutions](https://mlsanity.com/paper/2210.09394/) (2022)
- [Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization](https://mlsanity.com/paper/2309.08546/) (2023)
- [Self-Motivated Growing Neural Network for Adaptive Architecture via Local Structural Plasticity](https://mlsanity.com/paper/2512.12713/) (2025)
- [CULT: Continual Unsupervised Learning with Typicality-Based Environment Detection](https://mlsanity.com/paper/2207.08309/) (2022)
- [Data-Free Generative Replay for Class-Incremental Learning on Imbalanced Data](https://mlsanity.com/paper/2406.09052/) (2024)

## Abstract (arXiv)

Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages over conventional algorithms. In contrast to biological neural systems, which are capable of learning continuously, deep artificial networks have a limited ability for incorporating new information in an already trained network. As a result, methods for continuous learning are potentially highly impactful in enabling the application of deep networks to dynamic data sets. Here, inspired by the process of adult neurogenesis in the hippocampus, we explore the potential for adding new neurons to deep layers of artificial neural networks in order to facilitate their acquisition of novel information while preserving previously trained data representations. Our results on the MNIST handwritten digit dataset and the NIST SD 19 dataset, which includes lower and upper case letters and digits, demonstrate that neurogenesis is well suited for addressing the stability-plasticity dilemma that has long challenged adaptive machine learning algorithms.