mlsanityData to 7 Oct 2026

Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization

arXiv 2302.05440 ↗First posted 10 Feb 2023cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

"Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of backpropagation. Here, we first address compelling challenges related to the "forward-only" rules, which include reducing the performance gap with backpropagation and providing an analytical understanding of their dynamics. To this end, we show that the forward-only algorithm with top-down feedback is well-approximated by an "adaptive-feedback-alignment" algorithm, and we analytically track its performance during learning in a prototype high-dimensional setting. Then, we compare different versions of forward-only algorithms, focusing on the Forward-Forward and PEPITA frameworks, and we show that they share the same learning principles. Overall, our work unveils the connections between three key neuro-inspired learning rules, providing a link between "forward-only" algorithms, i.e., Forward-Forward and PEPITA, and an approximation of backpropagation, i.e., Feedback Alignment.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 33 of its 72 references that are in the map. First posted on arXiv 10 Feb 2023.

Builds on

33 of 72 references.

References · 33

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  3. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
  4. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  5. 2014Difference Target Propagationcited by 144
  6. 2019Training Neural Networks with Local Error Signalscited by 135
  7. 2015Statistical physics of inference: Thresholds and algorithmscited by 128
  8. 2019Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setupcited by 112
  9. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 100
  10. 2019Deep Learning without Weight Transportcited by 87
  11. 2021Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeedcited by 67
  12. 2022Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Passcited by 63
21 more
  1. 2022Hebbian Deep Learning Without Feedbackcited by 61
  2. 2015How Important is Weight Symmetry in Backpropagation?cited by 57
  3. 2020Training End-to-End Analog Neural Networks with Equilibrium Propagationcited by 57
  4. 2020Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architecturescited by 56
  5. 2022Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networkscited by 50
  6. 2020Local plasticity rules can learn deep representations using self-supervised contrastive predictionscited by 47
  7. 2020The Neural Coding Framework for Learning Generative Modelscited by 44
  8. 2018Biologically Motivated Algorithms for Propagating Local Target Representationscited by 43
  9. 2018Biologically-plausible learning algorithms can scale to large datasetscited by 37
  10. 2023The Predictive Forward-Forward Algorithmcited by 35
  11. 2018Feedback alignment in deep convolutional networkscited by 31
  12. 2021Credit Assignment in Neural Networks through Deep Feedback Controlcited by 28
  13. 2021Credit Assignment Through Broadcasting a Global Error Vectorcited by 19
  14. 2022The Influence of Learning Rule on Representation Dynamics in Wide Neural Networkscited by 19
  15. 2018Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learningcited by 16
  16. 2021Predictive Coding Can Do Exact Backpropagation on Convolutional and Recurrent Neural Networkscited by 14
  17. 2022Spike-based local synaptic plasticity: A survey of computational models and neuromorphic circuitscited by 13
  18. 2018Unsupervised Learning with Self-Organizing Spiking Neural Networkscited by 11
  19. 2022Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Imagescited by 10
  20. 2018Contrastive Hebbian Learning with Random Feedback Weightscited by 8
  21. 2021Online Training of Spiking Recurrent Neural Networks with Phase-Change Memory Synapsescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 4; 2026: 3.

13 papers.

Sort:

  1. 2023Backpropagation-free Training of Deep Physical Neural Networkscited by 42
  2. 2024Self-Contrastive Forward-Forward Algorithmcited by 15
  3. 2024Optimal Protocols for Continual Learning via Statistical Physics and Control Theorycited by 14
  4. 2024Beyond Backpropagation: Optimization with Multi-Tangent Forward Gradientscited by 6
  5. 2024LightFF: Lightweight Inference for Forward-Forward Algorithmcited by 3
  6. 2025A statistical physics framework for optimal learningcited by 3
  7. 2025Closed-Form Feedback-Free Learning with Forward Projectioncited by 2
  8. 2023Intrinsic Biologically Plausible Adversarial Robustnesscited by 1
  9. 2026Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional datacited by 1
  10. 2026HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithmcited by 1
  11. 2025Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Lossescited by 0
  12. 2025Forward Only Learning for Orthogonal Neural Networks of any Depthcited by 0
1 more
  1. 2026Can Local Learning Match Self-Supervised Backpropagation?cited by 0