mlsanityData to 14 Oct 2026

Interpreting learning in biological neural networks as zero-order optimization method

arXiv 2301.11777 ↗First posted 27 Jan 2023cs.LG cs.NE math.STcited by 4

Not on the map. Why?

Abstract (arXiv)

Recently, significant progress has been made regarding the statistical understanding of artificial neural networks (ANNs). ANNs are motivated by the functioning of the brain, but differ in several crucial aspects. In particular, the locality in the updating rule of the connection parameters in biological neural networks (BNNs) makes it biologically implausible that the learning of the brain is based on gradient descent. In this work, we look at the brain as a statistical method for supervised learning. The main contribution is to relate the local updating rule of the connection parameters in BNNs to a zero-order optimization method. It is shown that the expected values of the iterates implement a modification of gradient descent.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 10 of its 46 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

10 of 46 references.

References · 10

  1. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  2. 2018Deep Learning in Spiking Neural Networkscited by 263
  3. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
  4. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  5. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 198
  6. 2020A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learningcited by 162
  7. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 101
  8. 2020Predictive Coding Approximates Backprop along Arbitrary Computation Graphscited by 90
  9. 2021Predictive Coding: a Theoretical and Experimental Reviewcited by 77
  10. 2015How Important is Weight Symmetry in Backpropagation?cited by 58

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 1.

4 papers.

  1. 2023Is Learning in Biological Neural Networks based on Stochastic Gradient Descent? An analysis using stochastic processescited by 1
  2. 2023Hebbian learning inspired estimation of the linear regression parameters from queriescited by 1
  3. 2024Improving the Convergence Rates of Forward Gradient Descent with Repeated Samplingcited by 1
  4. 2025Memory Savings at What Cost? A Study of Alternatives to Backpropagationcited by 1