mlsanityData to 7 Oct 2026

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

arXiv 2006.12878 ↗First posted 23 Jun 2020stat.ML cs.LG cs.NEcited by 56

Not on the map. Why?

Abstract (arXiv)

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged. Alternative schemes have been devised; yet, under the constraint of synaptic asymmetry, none have scaled to modern deep learning tasks and architectures. Here, we challenge this perspective, and study the applicability of Direct Feedback Alignment to neural view synthesis, recommender systems, geometric learning, and natural language processing. In contrast with previous studies limited to computer vision tasks, our findings show that it successfully trains a large range of state-of-the-art deep learning architectures, with performance close to fine-tuned backpropagation. At variance with common beliefs, our work supports that challenging tasks can be tackled in the absence of weight transport.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 8; 2024: 11; 2025: 2; 2026: 15.

56 papers.

Sort:

By area:

  1. 2021Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systemscited by 125
  2. 2024Training of Physical Neural Networkscited by 47
  3. 2022Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillationscited by 40
  4. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  5. 2024Benchmarking Predictive Coding Networks -- Made Simplecited by 29
  6. 2022Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learningcited by 26
  7. 2022The Influence of Learning Rule on Representation Dynamics in Wide Neural Networkscited by 19
  8. 2021Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignmentcited by 17
  9. 2024Self-Contrastive Forward-Forward Algorithmcited by 15
  10. 2021Convergence and Alignment of Gradient Descent with Random Backpropagation Weightscited by 14
  11. 2021How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspectivecited by 13
  12. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
38 more
  1. 2022Is Bio-Inspired Learning Better than Backprop? Benchmarking Bio Learning vs. Backpropcited by 9
  2. 2023A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functionscited by 8
  3. 2023Improving equilibrium propagation without weight symmetry through Jacobian homeostasiscited by 8
  4. 2021Adversarial Robustness by Design through Analog Computing and Synthetic Gradientscited by 7
  5. 2022Low-Variance Forward Gradients using Direct Feedback Alignment and Momentumcited by 7
  6. 2023Block-local learning with probabilistic latent representationscited by 6
  7. 2024A Review of Neuroscience-Inspired Machine Learningcited by 6
  8. 2021Photonic Differential Privacy with Direct Feedback Alignmentcited by 5
  9. 2021How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorizationcited by 5
  10. 2024Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learningcited by 5
  11. 2026Test-Time Gradient Guidance of Flow Policies in Reinforcement Learningcited by 5
  12. 2022Scaling Laws Beyond Backpropagationcited by 4
  13. 2024Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignmentcited by 3
  14. 2021ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradientscited by 2
  15. 2024Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networkscited by 2
  16. 2024Training Spiking Neural Networks via Augmented Direct Feedback Alignmentcited by 2
  17. 2024Is All Learning (Natural) Gradient Descent?cited by 2
  18. 2024What should a neuron aim for? Designing local objective functions based on information theorycited by 2
  19. 2021Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in Deep Neural Networkscited by 1
  20. 2021Convergence Analysis and Implicit Regularization of Feedback Alignment for Deep Linear Networkscited by 1
  21. 2023Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networkscited by 1
  22. 2025Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learningcited by 1
  23. 2026Position: Zeroth-Order Optimization in Deep Learning Is Underexplored, Not Underpoweredcited by 1
  24. 2026Self-supervised local learning rules learn the hidden hierarchical structure of high-dimensional datacited by 1
  25. 2026Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networkscited by 1
  26. 2026Conditioned Direct Feedback Alignment via Activity and Error Geometrycited by 1
  27. 2021XConv: Low-memory stochastic backpropagation for convolutional layerscited by 0
  28. 2022Physical Deep Learning with Biologically Plausible Training Methodcited by 0
  29. 2022Emergent organization of receptive fields in networks of excitatory and inhibitory neuronscited by 0
  30. 2022A comparative study of back propagation and its alternatives on multilayer perceptronscited by 0
  31. 2023Learning with augmented target information: An alternative theory of Feedback Alignmentcited by 0
  32. 2023Random Feedback Alignment Algorithms to train Neural Networks: Why do they Align?cited by 0
  33. 2024Forward Direct Feedback Alignment for Online Gradient Estimates of Spiking Neural Networkscited by 0
  34. 2025Forward Only Learning for Orthogonal Neural Networks of any Depthcited by 0
  35. 2026SAL: Selective Adaptive Learning for Backpropagation-Free Training with Sparsificationcited by 0
  36. 2026Can Local Learning Match Self-Supervised Backpropagation?cited by 0
  37. 2026Score Broadcast and Decorrelation: A General Framework for Broadcast-Based Credit Assignmentcited by 0
  38. 2026Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Trainingcited by 0

50 most cited of 56 All 56 (JSON)