# Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment

arXiv: https://arxiv.org/abs/2012.06373 · HTML: https://mlsanity.com/paper/2012.06373/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment](https://mlsanity.com/paper/2111.06862/) (2021)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms](https://mlsanity.com/paper/2108.13446/) (2021)
- [Photonic Differential Privacy with Direct Feedback Alignment](https://mlsanity.com/paper/2106.03645/) (2021)
- [Scaling Laws Beyond Backpropagation](https://mlsanity.com/paper/2210.14593/) (2022)
- [First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models](https://mlsanity.com/paper/2311.05020/) (2023)
- [Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization](https://mlsanity.com/paper/2507.05583/) (2025)
- [Physical Deep Learning with Biologically Plausible Training Method](https://mlsanity.com/paper/2204.13991/) (2022)

## Abstract (arXiv)

The scaling hypothesis motivates the expansion of models past trillions of parameters as a path towards better performance. Recent significant developments, such as GPT-3, have been driven by this conjecture. However, as models scale-up, training them efficiently with backpropagation becomes difficult. Because model, pipeline, and data parallelism distribute parameters and gradients over compute nodes, communication is challenging to orchestrate: this is a bottleneck to further scaling. In this work, we argue that alternative training methods can mitigate these issues, and can inform the design of extreme-scale training hardware. Indeed, using a synaptically asymmetric method with a parallelizable backward pass, such as Direct Feedback Alignement, communication needs are drastically reduced. We present a photonic accelerator for Direct Feedback Alignment, able to compute random projections with trillions of parameters. We demonstrate our system on benchmark tasks, using both fully-connected and graph convolutional networks. Our hardware is the first architecture-agnostic photonic co-processor for training neural networks. This is a significant step towards building scalable hardware, able to go beyond backpropagation, and opening new avenues for deep learning.