Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule
Not on the map. Why?
Abstract (arXiv)
Backpropagation (BP) has been pivotal in advancing machine learning and remains essential in computational applications and comparative studies of biological and artificial neural networks. Despite its widespread use, the implementation of BP in the brain remains elusive, and its biological plausibility is often questioned due to inherent issues such as the need for symmetry of weights between forward and backward connections, and the requirement of distinct forward and backward phases of computation. Here, we introduce a novel neuroplasticity rule that offers a potential mechanism for implementing BP in the brain. Similar in general form to the classical Hebbian rule, this rule is based on the core principles of maintaining the balance of excitatory and inhibitory inputs as well as on retrograde signaling, and operates over three progressively slower timescales: neural firing, retrograde signaling, and neural plasticity. We hypothesize that each neuron possesses an internal state, termed credit, in addition to its firing rate. After achieving equilibrium in firing rates, neurons receive credits based on their contribution to the E-I balance of postsynaptic neurons through retrograde signaling. As the network's credit distribution stabilizes, connections from those presynaptic neurons are strengthened that significantly contribute to the balance of postsynaptic neurons. We demonstrate mathematically that our learning rule precisely replicates BP in layered neural networks without any approximations. Simulations on artificial neural networks reveal that this rule induces varying community structures in networks, depending on the learning rate. This simple theoretical framework presents a biologically plausible implementation of BP, with testable assumptions and predictions that may be evaluated through biological experiments.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 79 references that are in the map. First posted on arXiv 23 May 2024.
Builds on
20 of 79 references.
References · 20
- 2017Attention Is All You Needcited by 60,361
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 461
- 2015Deep Speech 2: End-to-End Speech Recognition in English and Mandarincited by 451
- 2018From Louvain to Leiden: guaranteeing well-connected communitiescited by 385
- 2019Financial Time Series Forecasting with Deep Learning : A Systematic Literature Review: 2005-2019cited by 273
- 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
- 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
- 2014Difference Target Propagationcited by 148
- 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 101
- 2019Deep Learning without Weight Transportcited by 87
- 2014How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagationcited by 79
8 more
- 2015How Important is Weight Symmetry in Backpropagation?cited by 58
- 2020A Theoretical Framework for Target Propagationcited by 57
- 2022Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillationscited by 42
- 2017Equivalence of Equilibrium Propagation and Recurrent Backpropagationcited by 32
- 2021Credit Assignment in Neural Networks through Deep Feedback Controlcited by 29
- 2019Learning to solve the credit assignment problemcited by 27
- 2021Credit Assignment Through Broadcasting a Global Error Vectorcited by 19
- 2022The least-control principle for local learning at equilibriumcited by 16
Built on by
Not cited yet.