mlsanityData to 14 Oct 2026

Neural Architecture Search with Mixed Bio-inspired Learning Rules

arXiv 2507.13485 ↗First posted 17 Jul 2025cs.NE cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Bio-inspired neural networks are attractive for their adversarial robustness, energy frugality, and closer alignment with cortical physiology, yet they often lag behind back-propagation (BP) based models in accuracy and ability to scale. We show that allowing the use of different bio-inspired learning rules in different layers, discovered automatically by a tailored neural-architecture-search (NAS) procedure, bridges this gap. Starting from standard NAS baselines, we enlarge the search space to include bio-inspired learning rules and use NAS to find the best architecture and learning rule to use in each layer. We show that neural networks that use different bio-inspired learning rules for different layers have better accuracy than those that use a single rule across all the layers. The resulting NN that uses a mix of bio-inspired learning rules sets new records for bio-inspired models: 95.16% on CIFAR-10, 76.48% on CIFAR-100, 43.42% on ImageNet16-120, and 60.51% top-1 on ImageNet. In some regimes, they even surpass comparable BP-based networks while retaining their robustness advantages. Our results suggest that layer-wise diversity in learning rules allows better scalability and accuracy, and motivates further research on mixing multiple bio-inspired learning rules in the same network.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 54 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

41 of 54 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  6. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  7. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,150
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  10. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  12. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
29 more
  1. 2017One pixel attack for fooling deep neural networkscited by 582
  2. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  3. 2016The CMA Evolution Strategy: A Tutorialcited by 377
  4. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  5. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  6. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 266
  7. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  8. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 260
  9. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  10. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  11. 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
  12. 2018Differentiable plasticity: training plastic neural networks with backpropagationcited by 84
  13. 2016Towards an integration of deep learning and neurosciencecited by 78
  14. 2020DrNAS: Dirichlet Neural Architecture Searchcited by 73
  15. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  16. 2020DSNAS: Direct Neural Architecture Search without Parameter Retrainingcited by 69
  17. 2019BayesNAS: A Bayesian Approach for Neural Architecture Searchcited by 68
  18. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  19. 2022Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Passcited by 63
  20. 2022Hebbian Deep Learning Without Feedbackcited by 61
  21. 2015How Important is Weight Symmetry in Backpropagation?cited by 58
  22. 2022Neural Architecture Search for Spiking Neural Networkscited by 53
  23. 2016CMA-ES for Hyperparameter Optimization of Deep Neural Networkscited by 48
  24. 2021SoftHebb: Bayesian Inference in Unsupervised Hebbian Soft Winner-Take-All Networkscited by 30
  25. 2022FastHebb: Scaling Hebbian Training of Deep Neural Networks to ImageNet Levelcited by 11
  26. 2021iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretizationcited by 10
  27. 2021Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithmscited by 9
  28. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
  29. 2023Robust Neural Architecture Searchcited by 5

Built on by

Not cited yet.