Towards Brain Inspired Design for Addressing the Shortcomings of ANNs
Not on the map. Why?
Abstract (arXiv)
As our understanding of the mechanisms of brain function is enhanced, the value of insights gained from neuroscience to the development of AI algorithms deserves further consideration. Here, we draw parallels with an existing tree-based ANN architecture and a recent neuroscience study[27] arguing that the error-based organization of neurons in the cerebellum that share a preference for a personalized view of the entire error space, may account for several desirable features of behavior and learning. We then analyze the learning behavior and characteristics of the model under varying scenarios to gauge the potential benefits of a similar mechanism in ANN. Our empirical results suggest that having separate populations of neurons with personalized error views can enable efficient learning under class imbalance and limited data, and reduce the susceptibility to unintended shortcut strategies, leading to improved generalization. This work highlights the potential of translating the learning machinery of the brain into the design of a new generation of ANNs and provides further credence to the argument that biologically inspired AI may hold the key to overcoming the shortcomings of ANNs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 30 Jun 2023.
Builds on
21 of 38 references.
References · 21
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2016Wide Residual Networkscited by 3,070
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2020Shortcut Learning in Deep Neural Networkscited by 1,969
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
- 2019Natural Adversarial Examplescited by 1,408
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
- 2018Robustness May Be at Odds with Accuracycited by 880
- 2015Deep Learning and the Information Bottleneck Principlecited by 802
9 more
- 2017Deep Mutual Learningcited by 628
- 2017Opening the Black Box of Deep Neural Networks via Informationcited by 503
- 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
- 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
- 2018Imbalanced Deep Learning by Minority Class Incremental Rectificationcited by 65
- 2020Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbationscited by 55
- 2019Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracycited by 49
- 2021Combining Diverse Feature Priorscited by 12
- 2021Machine learning with limited datacited by 2
Built on by
Citing papers per year: 2023: 1.
1 paper.