# How Important is Weight Symmetry in Backpropagation?

arXiv: https://arxiv.org/abs/1510.05067 · HTML: https://mlsanity.com/paper/1510.05067/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2015.

## Built on by (most cited first, 30 of 58)

- [Scaling Forward Gradient With Local Losses](https://mlsanity.com/paper/2210.03310/) (2022)
- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [Can Forward Gradient Match Backpropagation?](https://mlsanity.com/paper/2306.06968/) (2023)
- [Signal Propagation: A Framework for Learning and Inference In a Forward Pass](https://mlsanity.com/paper/2204.01723/) (2022)
- [Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligence](https://mlsanity.com/paper/2106.01288/) (2021)
- [The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware](https://mlsanity.com/paper/2106.07030/) (2021)
- [The Influence of Learning Rule on Representation Dynamics in Wide Neural Networks](https://mlsanity.com/paper/2210.02157/) (2022)
- [Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathways](https://mlsanity.com/paper/2210.16414/) (2022)
- [How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective](https://mlsanity.com/paper/2106.08453/) (2021)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Hierarchical Reasoning Models: Perspectives and Misconceptions](https://mlsanity.com/paper/2510.00355/) (2025)
- [Applications of the Free Energy Principle to Machine Learning and Neuroscience](https://mlsanity.com/paper/2107.00140/) (2021)
- [Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithms](https://mlsanity.com/paper/2108.13446/) (2021)
- [Improving equilibrium propagation without weight symmetry through Jacobian homeostasis](https://mlsanity.com/paper/2309.02214/) (2023)
- [Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networks](https://mlsanity.com/paper/2109.15089/) (2021)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods](https://mlsanity.com/paper/2101.03419/) (2021)
- [Pretraining with Random Noise for Fast and Robust Learning without Weight Transport](https://mlsanity.com/paper/2405.16731/) (2024)
- [Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithm](https://mlsanity.com/paper/2502.20411/) (2025)
- [Interpreting learning in biological neural networks as zero-order optimization method](https://mlsanity.com/paper/2301.11777/) (2023)
- [Synaptic Weight Distributions Depend on the Geometry of Plasticity](https://mlsanity.com/paper/2305.19394/) (2023)
- [Towards Interpretable Deep Local Learning with Successive Gradient Reconciliation](https://mlsanity.com/paper/2406.05222/) (2024)
- [Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetry](https://mlsanity.com/paper/2306.04810/) (2023)
- [LightFF: Lightweight Inference for Forward-Forward Algorithm](https://mlsanity.com/paper/2404.05241/) (2024)
- [Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networks](https://mlsanity.com/paper/2407.12516/) (2024)
- [Deep Layer-wise Networks Have Closed-Form Weights](https://mlsanity.com/paper/2202.01210/) (2022)
- [Brain-like Flexible Visual Inference by Harnessing Feedback-Feedforward Alignment](https://mlsanity.com/paper/2310.20599/) (2023)
- [Training Large Neural Networks With Low-Dimensional Error Feedback](https://mlsanity.com/paper/2502.20580/) (2025)
- [Overcoming Rank Collapse in Feedback Alignment](https://mlsanity.com/paper/2606.11123/) (2026)
- [Tourbillon: a Physically Plausible Neural Architecture](https://mlsanity.com/paper/2107.06424/) (2021)

## Abstract (arXiv)

Gradient backpropagation (BP) requires symmetric feedforward and feedback connections -- the same weights must be used for forward and backward passes. This "weight transport problem" (Grossberg 1987) is thought to be one of the main reasons to doubt BP's biologically plausibility. Using 15 different classification datasets, we systematically investigate to what extent BP really depends on weight symmetry. In a study that turned out to be surprisingly similar in spirit to Lillicrap et al.'s demonstration (Lillicrap et al. 2014) but orthogonal in its results, our experiments indicate that: (1) the magnitudes of feedback weights do not matter to performance (2) the signs of feedback weights do matter -- the more concordant signs between feedforward and their corresponding feedback connections, the better (3) with feedback weights having random magnitudes and 100% concordant signs, we were able to achieve the same or even better performance than SGD. (4) some normalizations/stabilizations are indispensable for such asymmetric BP to work, namely Batch Normalization (BN) (Ioffe and Szegedy 2015) and/or a "Batch Manhattan" (BM) update rule.