# Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

arXiv: https://arxiv.org/abs/2006.12878 · HTML: https://mlsanity.com/paper/2006.12878/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2020.

## Built on by (most cited first, 30 of 56)

- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations](https://mlsanity.com/paper/2209.00530/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Benchmarking Predictive Coding Networks -- Made Simple](https://mlsanity.com/paper/2407.01163/) (2024)
- [Backpropagation at the Infinitesimal Inference Limit of Energy-Based Models: Unifying Predictive Coding, Equilibrium Propagation, and Contrastive Hebbian Learning](https://mlsanity.com/paper/2206.02629/) (2022)
- [The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks](https://mlsanity.com/paper/2210.02157/) (2022)
- [Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment](https://mlsanity.com/paper/2111.06862/) (2021)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Convergence and Alignment of Gradient Descent with Random Backpropagation Weights](https://mlsanity.com/paper/2106.06044/) (2021)
- [How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective](https://mlsanity.com/paper/2106.08453/) (2021)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Is Bio-Inspired Learning Better than Backprop? Benchmarking Bio Learning vs. Backprop](https://mlsanity.com/paper/2212.04614/) (2022)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [A General Framework for Interpretable Neural Learning based on Local Information-Theoretic Goal Functions](https://mlsanity.com/paper/2306.02149/) (2023)
- [Adversarial Robustness by Design through Analog Computing and Synthetic Gradients](https://mlsanity.com/paper/2101.02115/) (2021)
- [Low-Variance Forward Gradients using Direct Feedback Alignment and Momentum](https://mlsanity.com/paper/2212.07282/) (2022)
- [Block-local learning with probabilistic latent representations](https://mlsanity.com/paper/2305.14974/) (2023)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning](https://mlsanity.com/paper/2606.11087/) (2026)
- [Photonic Differential Privacy with Direct Feedback Alignment](https://mlsanity.com/paper/2106.03645/) (2021)
- [How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorization](https://mlsanity.com/paper/2111.08706/) (2021)
- [Counter-Current Learning: A Biologically Plausible Dual Network Approach for Deep Learning](https://mlsanity.com/paper/2409.19841/) (2024)
- [Scaling Laws Beyond Backpropagation](https://mlsanity.com/paper/2210.14593/) (2022)
- [Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks](https://mlsanity.com/paper/2407.12516/) (2024)
- [Streamlined optical training of large-scale modern deep learning architectures with direct feedback alignment](https://mlsanity.com/paper/2409.12965/) (2024)
- [What should a neuron aim for? Designing local objective functions based on information theory](https://mlsanity.com/paper/2412.02482/) (2024)
- [ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients](https://mlsanity.com/paper/2108.04217/) (2021)
- [Training Spiking Neural Networks via Augmented Direct Feedback Alignment](https://mlsanity.com/paper/2409.07776/) (2024)
- [Is All Learning (Natural) Gradient Descent?](https://mlsanity.com/paper/2409.16422/) (2024)

## Abstract (arXiv)

Despite being the workhorse of deep learning, the backpropagation algorithm is no panacea. It enforces sequential layer updates, thus preventing efficient parallelization of the training process. Furthermore, its biological plausibility is being challenged. Alternative schemes have been devised; yet, under the constraint of synaptic asymmetry, none have scaled to modern deep learning tasks and architectures. Here, we challenge this perspective, and study the applicability of Direct Feedback Alignment to neural view synthesis, recommender systems, geometric learning, and natural language processing. In contrast with previous studies limited to computer vision tasks, our findings show that it successfully trains a large range of state-of-the-art deep learning architectures, with performance close to fine-tuned backpropagation. At variance with common beliefs, our work supports that challenging tasks can be tackled in the absence of weight transport.