# Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms

arXiv: https://arxiv.org/abs/2108.13446 · HTML: https://mlsanity.com/paper/2108.13446/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 30 Aug 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Torchattacks: A PyTorch Repository for Adversarial Attacks](https://mlsanity.com/paper/2010.01950/) (2020)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [Deep Learning without Weight Transport](https://mlsanity.com/paper/1904.05391/) (2019)
- [How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation](https://mlsanity.com/paper/1407.7906/) (2014)
- [How Important is Weight Symmetry in Backpropagation?](https://mlsanity.com/paper/1510.05067/) (2015)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [Biologically-plausible learning algorithms can scale to large datasets](https://mlsanity.com/paper/1811.03567/) (2018)
- [Feedback alignment in deep convolutional networks](https://mlsanity.com/paper/1812.06488/) (2018)
- [Two Routes to Scalable Credit Assignment without Weight Symmetry](https://mlsanity.com/paper/2003.01513/) (2020)
- [Deriving Differential Target Propagation from Iterating Approximate Inverses](https://mlsanity.com/paper/2007.15139/) (2020)
- [Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"](https://mlsanity.com/paper/1907.00895/) (2019)
- [Hardware Beyond Backpropagation: a Photonic Co-Processor for Direct Feedback Alignment](https://mlsanity.com/paper/2012.06373/) (2020)
- [Adversarial Robustness by Design through Analog Computing and Synthetic Gradients](https://mlsanity.com/paper/2101.02115/) (2021)
- [On the Adversarial Robustness of Neural Networks without Weight Transport](https://mlsanity.com/paper/1908.03560/) (2019)
- [ROPUST: Improving Robustness through Fine-tuning with Photonic Processors and Synthetic Gradients](https://mlsanity.com/paper/2108.04217/) (2021)

## Built on by (most cited first, 9 of 9)

- [Block-local learning with probabilistic latent representations](https://mlsanity.com/paper/2305.14974/) (2023)
- [Scaling Laws Beyond Backpropagation](https://mlsanity.com/paper/2210.14593/) (2022)
- [Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networks](https://mlsanity.com/paper/2306.01870/) (2023)
- [Deep Learning without Weight Symmetry](https://mlsanity.com/paper/2405.20594/) (2024)
- [Feedback Alignment Meets Low-Rank Manifolds: A Structured Recipe for Local Learning](https://mlsanity.com/paper/2510.25594/) (2025)
- [Random Feedback Alignment Algorithms to train Neural Networks: Why do they Align?](https://mlsanity.com/paper/2306.02325/) (2023)
- [Neural Architecture Search with Mixed Bio-inspired Learning Rules](https://mlsanity.com/paper/2507.13485/) (2025)
- [Realist and Pluralist Conceptions of Intelligence and Their Implications on AI Research](https://mlsanity.com/paper/2511.15282/) (2025)
- [Common-Mode Collapse and Recovery in Direct Feedback Alignment](https://mlsanity.com/paper/2609.31589/) (2026)

## Abstract (arXiv)

Backpropagation is the default algorithm for training deep neural networks due to its simplicity, efficiency and high convergence rate. However, its requirements make it impossible to be implemented in a human brain. In recent years, more biologically plausible learning methods have been proposed. Some of these methods can match backpropagation accuracy, and simultaneously provide other extra benefits such as faster training on specialized hardware (e.g., ASICs) or higher robustness against adversarial attacks. While the interest in the field is growing, there is a necessity for open-source libraries and toolkits to foster research and benchmark algorithms. In this paper, we present BioTorch, a software framework to create, train, and benchmark biologically motivated neural networks. In addition, we investigate the performance of several feedback alignment methods proposed in the literature, thereby unveiling the importance of the forward and backward weight initialization and optimizer choice. Finally, we provide a novel robustness study of these methods against state-of-the-art white and black-box adversarial attacks.