mlsanityData to 7 Oct 2026

Credit Assignment in Neural Networks through Deep Feedback Control

arXiv 2106.07887 ↗First posted 15 Jun 2021cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

The success of deep learning sparked interest in whether the brain learns by using similar techniques for assigning credit to each synaptic weight for its contribution to the network output. However, the majority of current attempts at biologically-plausible learning methods are either non-local in time, require highly specific connectivity motives, or have no clear link to any known mathematical optimization method. Here, we introduce Deep Feedback Control (DFC), a new learning method that uses a feedback controller to drive a deep neural network to match a desired output target and whose control signal can be used for credit assignment. The resulting learning rule is fully local in space and time and approximates Gauss-Newton optimization for a wide range of feedback connectivity patterns. To further underline its biological plausibility, we relate DFC to a multi-compartment model of cortical pyramidal neurons with a local voltage-dependent synaptic plasticity rule, consistent with recent theories of dendritic processing. By combining dynamical system theory with mathematical optimization theory, we provide a strong theoretical foundation for DFC that we corroborate with detailed results on toy experiments and standard computer-vision benchmarks.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 30 of its 78 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

30 of 78 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  5. 2018Tune: A Research Platform for Distributed Model Selection and Trainingcited by 395
  6. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  7. 2014Difference Target Propagationcited by 144
  8. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 139
  9. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 100
  10. 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 96
  11. 2019Deep Learning without Weight Transportcited by 87
  12. 2014How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagationcited by 77
18 more
  1. 2019Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networkscited by 76
  2. 2016Towards deep learning with segregated dendritescited by 71
  3. 2015Towards Biologically Plausible Deep Learningcited by 66
  4. 2020A Theoretical Framework for Target Propagationcited by 56
  5. 2019Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problemscited by 40
  6. 2018Feedback alignment in deep convolutional networkscited by 31
  7. 2019Learning to solve the credit assignment problemcited by 27
  8. 2019Direct Feedback Alignment with Sparse Connections for Local Learningcited by 26
  9. 2019Principled Training of Neural Networks with Direct Feedback Alignmentcited by 23
  10. 2020Two Routes to Scalable Credit Assignment without Weight Symmetrycited by 19
  11. 2018Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learningcited by 16
  12. 2020Deriving Differential Target Propagation from Iterating Approximate Inversescited by 15
  13. 2017Predicting non-linear dynamics by stable local learning in a recurrent spiking neural networkcited by 12
  14. 2017The brain as an efficient and robust adaptive learnercited by 11
  15. 2019Spike-based causal inference for weight alignmentcited by 10
  16. 2020Biological credit assignment through dynamic inversion of feedforward networkscited by 10
  17. 2017Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rulescited by 3
  18. 2020A biologically plausible neural network for local supervision in cortical microcircuitscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 5; 2024: 5; 2025: 2; 2026: 7.

28 papers.

Sort:

By area:

  1. 2022The least-control principle for local learning at equilibriumcited by 16
  2. 2022A Theoretical Framework for Inference and Learning in Predictive Coding Networkscited by 16
  3. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
  4. 2022Minimizing Control for Credit Assignment with Strong Feedbackcited by 10
  5. 2022Learning efficient backprojections across cortical hierarchies in real timecited by 8
  6. 2023Improving equilibrium propagation without weight symmetry through Jacobian homeostasiscited by 8
  7. 2022Bio-Inspired, Task-Free Continual Learning through Activity Regularizationcited by 7
  8. 2024Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learningcited by 5
  9. 2022Thalamus: a brain-inspired algorithm for biologically-plausible continual learning and disentangled representationscited by 4
  10. 2021Target Propagation via Regularized Inversioncited by 3
  11. 2022Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rulescited by 3
  12. 2024LightFF: Lightweight Inference for Forward-Forward Algorithmcited by 3
16 more
  1. 2023Duality Principle and Biologically Plausible Learning: Connecting the Representer Theorem and Hebbian Learningcited by 2
  2. 2025Toward Practical Equilibrium Propagation: Brain-inspired Recurrent Neural Network with Feedback Regulation and Residual Connectionscited by 2
  3. 2026Continual Learning through Control Minimizationcited by 2
  4. 2026A feedback control optimizer for online and hardware-aware training of Spiking Neural Networkscited by 2
  5. 2023Intrinsic Biologically Plausible Adversarial Robustnesscited by 1
  6. 2024A Unified Framework for Neural Computation and Learning Over Timecited by 1
  7. 2025Spike-based alignment learning solves the weight transport problemcited by 1
  8. 2026Credit Assignment via Neural Manifold Noise Correlationcited by 1
  9. 2026Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional datacited by 1
  10. 2022Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Networkcited by 0
  11. 2023Dis-inhibitory neuronal circuits can control the sign of synaptic plasticitycited by 0
  12. 2024Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rulecited by 0
  13. 2024Stimulus-to-Stimulus Learning in RNNs with Cortical Inductive Biasescited by 0
  14. 2026Mixed-signal implementation of feedback-control optimizer for single-layer Spiking Neural Networkscited by 0
  15. 2026Graph-based Target Back-Propagation for Context Adaptation in Multi-LLM Agentic Systemscited by 0
  16. 2026The Ups and Downs of Backprop Weightscited by 0