mlsanityData to 14 Oct 2026

Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learning

arXiv 2503.21419 ↗First posted 27 Mar 2025cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Artificial Intelligence (AI) has achieved new levels of performance and spread in public usage with the rise of deep neural networks (DNNs). Initially inspired by human neurons and their connections, NNs have become the foundation of AI models for many advanced architectures. However, some of the most integral processes in the human brain, particularly neurogenesis and neuroplasticity in addition to the more spread neuroapoptosis have largely been ignored in DNN architecture design. Instead, contemporary AI development predominantly focuses on constructing advanced frameworks, such as large language models, which retain a static structure of neural connections during training and inference. In this light, we explore how neurogenesis, neuroapoptosis, and neuroplasticity can inspire future AI advances. Specifically, we examine analogous activities in artificial NNs, introducing the concepts of ``dropin'' for neurogenesis and revisiting ``dropout'' and structural pruning for neuroapoptosis. We additionally suggest neuroplasticity combining the two for future large NNs in ``life-long learning'' settings following the biological inspiration. We conclude by advocating for greater research efforts in this interdisciplinary domain and identifying promising directions for future exploration.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 82 of its 168 references that are in the map. First posted on arXiv 27 Mar 2025.

Builds on

82 of 168 references.

References · 82

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  11. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  12. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
38 more
  1. 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 656
  2. 2020What is the State of Neural Network Pruning?cited by 583
  3. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  4. 2018A Survey on Deep Transfer Learningcited by 419
  5. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  6. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 274
  7. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 256
  8. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  9. 2014Efficient Object Localization Using Convolutional Networkscited by 227
  10. 2017Concrete Dropoutcited by 227
  11. 2016Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learningcited by 209
  12. 2018Online Learning: A Comprehensive Surveycited by 205
  13. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
  14. 2013Maxout Networkscited by 187
  15. 2018Reinforced Continual Learningcited by 186
  16. 2018Stochastic Activation Pruning for Robust Adversarial Defensecited by 171
  17. 2018SpotTune: Transfer Learning through Adaptive Fine-tuningcited by 168
  18. 2016Information Dropout: Learning Optimal Representations Through Noisy Computationcited by 132
  19. 2020Layer-adaptive sparsity for the Magnitude-based Pruningcited by 126
  20. 2018An introduction to domain adaptation and transfer learningcited by 112
  21. 2023A Definition of Continual Reinforcement Learningcited by 101
  22. 2024LLaMA Pro: Progressive LLaMA with Block Expansioncited by 101
  23. 2013Dropout Training as Adaptive Regularizationcited by 93
  24. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 88
  25. 2017Online Deep Learning: Learning Deep Neural Networks on the Flycited by 86
  26. 2024Continual Learning and Catastrophic Forgettingcited by 77
  27. 2021Essentials for Class Incremental Learningcited by 57
  28. 2022An Efficient Domain-Incremental Learning Approach to Drive in All Weather Conditionscited by 54
  29. 2017Curriculum Dropoutcited by 50
  30. 2016Pruning Convolutional Neural Networks for Resource Efficient Inferencecited by 48
  31. 2019A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networkscited by 38
  32. 2023Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzlecited by 37
  33. 2019The Information Complexity of Learning Tasks, their Structure and their Distancecited by 36
  34. 2020A Gradient Flow Framework For Analyzing Network Pruningcited by 36
  35. 2015Transfer Learning for Speech and Language Processingcited by 35
  36. 2024LoRA Dropout as a Sparsity Regularizer for Overfitting Controlcited by 35
  37. 2020A Survey on Transfer Learning in Natural Language Processingcited by 33
  38. 2022FairGRAPE: Fairness-aware GRAdient Pruning mEthod for Face Attribute Classificationcited by 30

50 most cited of 82 All 82 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Large Language Models for Depression Recognition in Spoken Language Integrating Psychological Knowledgecited by 3
  2. 2025Dynamic Nested Hierarchies: Pioneering Self-Evolution in Machine Learning Architectures for Lifelong Intelligencecited by 1
  3. 2026Enhancing Efficiency and Performance in Deepfake Audio Detection through Neuron-level Dropin & Neuroplasticity Mechanismscited by 0