mlsanityData to 7 Oct 2026

Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks

arXiv 2301.09245 ↗First posted 23 Jan 2023cs.NE cs.AI cs.ETcited by 6

Not on the map. Why?

Abstract (arXiv)

Throughout history, the development of artificial intelligence, particularly artificial neural networks, has been open to and constantly inspired by the increasingly deepened understanding of the brain, such as the inspiration of neocognitron, which is the pioneering work of convolutional neural networks. Per the motives of the emerging field: NeuroAI, a great amount of neuroscience knowledge can help catalyze the next generation of AI by endowing a network with more powerful capabilities. As we know, the human brain has numerous morphologically and functionally different neurons, while artificial neural networks are almost exclusively built on a single neuron type. In the human brain, neuronal diversity is an enabling factor for all kinds of biological intelligent behaviors. Since an artificial network is a miniature of the human brain, introducing neuronal diversity should be valuable in terms of addressing those essential problems of artificial networks such as efficiency, interpretability, and memory. In this Primer, we first discuss the preliminaries of biological neuronal diversity and the characteristics of information transmission and processing in a biological neuron. Then, we review studies of designing new neurons for artificial networks. Next, we discuss what gains can neuronal diversity bring into artificial networks and exemplary applications in several important fields. Lastly, we discuss the challenges and future directions of neuronal diversity to explore the potential of NeuroAI.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 48 of its 212 references that are in the map. First posted on arXiv 23 Jan 2023.

Builds on

48 of 212 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  9. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  10. 2016Learning without Forgettingcited by 2,465
  11. 2016Progressive Neural Networkscited by 1,332
  12. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
36 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  2. 2017Continual Learning with Deep Generative Replaycited by 1,152
  3. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  4. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  5. 2016The Mythos of Model Interpretabilitycited by 716
  6. 2019AutoML: A Survey of the State-of-the-Artcited by 388
  7. 2016Exponential expressivity in deep neural networks through transient chaoscited by 260
  8. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  9. 2018Going Deeper in Spiking Neural Networks: VGG and Residual Architecturescited by 236
  10. 2016Training Deep Spiking Neural Networks using Backpropagationcited by 203
  11. 2016Deep Pyramidal Residual Networkscited by 163
  12. 2015On the Expressive Power of Deep Learning: A Tensor Analysiscited by 97
  13. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88
  14. 2016Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Unitscited by 86
  15. 2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
  16. 2014On the Computational Efficiency of Training Neural Networkscited by 71
  17. 2020Deep Polynomial Neural Networkscited by 63
  18. 2021A Survey of Complex-Valued Neural Networkscited by 61
  19. 2016Noisy Activation Functionscited by 53
  20. 2017Dense Associative Memory is Robust to Adversarial Inputscited by 45
  21. 2015Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortexcited by 44
  22. 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 42
  23. 2017Non-linear Convolution Filters for CNN-based Learningcited by 38
  24. 2019Quadratic Autoencoder (Q-AE) for Low-dose CT Denoisingcited by 35
  25. 2015Deep Learning with S-shaped Rectified Linear Activation Unitscited by 26
  26. 2022QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Explorationcited by 21
  27. 2018On a Sparse Shortcut Topology of Artificial Neural Networkscited by 20
  28. 2017A New Type of Neurons for Machine Learningcited by 16
  29. 2020Dendrite Net: A White-Box Module for Classification, Regression, and System Identificationcited by 11
  30. 2019Natural-Logarithm-Rectified Activation Function in Convolutional Neural Networkscited by 8
  31. 2022Attention-embedded Quadratic Network (Qttention) for Effective and Interpretable Bearing Fault Diagnosiscited by 8
  32. 2021Towards Understanding Theoretical Advantages of Complex-Reaction Networkscited by 6
  33. 2020Non-linear Neurons with Human-like Apical Dendrite Activationscited by 5
  34. 2022Quadratic Neuron-empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detectioncited by 5
  35. 2018Deep learning improved by biological activation functionscited by 3
  36. 2021Kuramoto model for populations of quadratic integrate-and-fire neurons with chemical and electrical couplingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 2; 2026: 1.

6 papers.

  1. 2023One Neuron Saved Is One Neuron Earned: On Parametric Efficiency of Quadratic Networkscited by 6
  2. 2024No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networkscited by 3
  3. 2024Augmenting learning in neuro-embodied systems through neurobiological first principlescited by 2
  4. 2025Improving the adaptive and continuous learning capabilities of artificial neural networks: Lessons from multi-neuromodulatory dynamicscited by 2
  5. 2025NeuronSeek: On Stability and Expressivity of Task-driven Neuronscited by 0
  6. 2026Beyond Weighted Summation: Learnable Nonlinear Aggregation Functions for Robust Artificial Neuronscited by 0