mlsanityData to 14 Oct 2026

How Important is Weight Symmetry in Backpropagation?

arXiv 1510.05067 ↗First posted 17 Oct 2015cs.LGcited by 58

Not on the map. Why?

Abstract (arXiv)

Gradient backpropagation (BP) requires symmetric feedforward and feedback connections -- the same weights must be used for forward and backward passes. This "weight transport problem" (Grossberg 1987) is thought to be one of the main reasons to doubt BP's biologically plausibility. Using 15 different classification datasets, we systematically investigate to what extent BP really depends on weight symmetry. In a study that turned out to be surprisingly similar in spirit to Lillicrap et al.'s demonstration (Lillicrap et al. 2014) but orthogonal in its results, our experiments indicate that: (1) the magnitudes of feedback weights do not matter to performance (2) the signs of feedback weights do matter -- the more concordant signs between feedforward and their corresponding feedback connections, the better (3) with feedback weights having random magnitudes and 100% concordant signs, we were able to achieve the same or even better performance than SGD. (4) some normalizations/stabilizations are indispensable for such asymmetric BP to work, namely Batch Normalization (BN) (Ioffe and Szegedy 2015) and/or a "Batch Manhattan" (BM) update rule.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Oct 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 7; 2023: 10; 2024: 11; 2025: 12; 2026: 7.

58 papers.

Sort:

By area:

  1. 2022Scaling Forward Gradient With Local Lossescited by 65
  2. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  3. 2023Can Forward Gradient Match Backpropagation?cited by 26
  4. 2022Signal Propagation: A Framework for Learning and Inference In a Forward Passcited by 24
  5. 2021Bottom-up and top-down approaches for the design of neuromorphic processing systems: Tradeoffs and synergies between natural and artificial intelligencecited by 21
  6. 2021The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardwarecited by 19
  7. 2022The Influence of Learning Rule on Representation Dynamics in Wide Neural Networkscited by 19
  8. 2022Meta-Learning Biologically Plausible Plasticity Rules with Random Feedback Pathwayscited by 14
  9. 2021How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspectivecited by 13
  10. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
  11. 2025Hierarchical Reasoning Models: Perspectives and Misconceptionscited by 13
  12. 2021Applications of the Free Energy Principle to Machine Learning and Neurosciencecited by 9
38 more
  1. 2021Benchmarking the Accuracy and Robustness of Feedback Alignment Algorithmscited by 9
  2. 2023Improving equilibrium propagation without weight symmetry through Jacobian homeostasiscited by 9
  3. 2021Biologically Plausible Training Mechanisms for Self-Supervised Learning in Deep Networkscited by 6
  4. 2024A Review of Neuroscience-Inspired Machine Learningcited by 6
  5. 2021Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methodscited by 5
  6. 2024Pretraining with Random Noise for Fast and Robust Learning without Weight Transportcited by 5
  7. 2025Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithmcited by 5
  8. 2023Interpreting learning in biological neural networks as zero-order optimization methodcited by 4
  9. 2023Synaptic Weight Distributions Depend on the Geometry of Plasticitycited by 4
  10. 2024Towards Interpretable Deep Local Learning with Successive Gradient Reconciliationcited by 4
  11. 2023Correlative Information Maximization: A Biologically Plausible Approach to Supervised Deep Neural Networks without Weight Symmetrycited by 3
  12. 2024LightFF: Lightweight Inference for Forward-Forward Algorithmcited by 3
  13. 2024Online Pseudo-Zeroth-Order Training of Neuromorphic Spiking Neural Networkscited by 3
  14. 2022Deep Layer-wise Networks Have Closed-Form Weightscited by 2
  15. 2023Brain-like Flexible Visual Inference by Harnessing Feedback-Feedforward Alignmentcited by 2
  16. 2025Training Large Neural Networks With Low-Dimensional Error Feedbackcited by 2
  17. 2026Overcoming Rank Collapse in Feedback Alignmentcited by 2
  18. 2021Tourbillon: a Physically Plausible Neural Architecturecited by 1
  19. 2021BioLCNet: Reward-modulated Locally Connected Spiking Neural Networkscited by 1
  20. 2021Information Bottleneck-Based Hebbian Learning Rule Naturally Ties Working Memory and Synaptic Updatescited by 1
  21. 2022A Computational Framework of Cortical Microcircuits Approximates Sign-concordant Random Backpropagationcited by 1
  22. 2023Implicit Regularization in Feedback Alignment Learning Mechanisms for Neural Networkscited by 1
  23. 2024Learning Using a Single Forward Passcited by 1
  24. 2024NeuroFlux: Memory-Efficient CNN Training Using Adaptive Local Learningcited by 1
  25. 2024Deep Learning without Weight Symmetrycited by 1
  26. 2024Oja's plasticity rule overcomes several challenges of training neural networks under biological constraintscited by 1
  27. 2025Spike-based alignment learning solves the weight transport problemcited by 1
  28. 2025One-Time Soft Alignment Enables Resilient Learning without Weight Transportcited by 1
  29. 2026Three factor delay learning rules for spiking neural networkscited by 1
  30. 2026Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream Networkscited by 1
  31. 2021Efficient Training Convolutional Neural Networks on Edge Devices with Gradient-pruned Sign-symmetric Feedback Alignmentcited by 0
  32. 2022Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Networkcited by 0
  33. 2023Zenkai -- Framework For Exploring Beyond Backpropagationcited by 0
  34. 2024Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rulecited by 0
  35. 2024Protecting Feed-Forward Networks from Adversarial Attacks Using Predictive Codingcited by 0
  36. 2025Sign-Symmetry Learning Rules are Robust Fine-Tunerscited by 0
  37. 2025Noise-based reward-modulated learningcited by 0
  38. 2025Frozen Backpropagation: Relaxing Weight Symmetry in Deep Spiking Neural Networkscited by 0

50 most cited of 58 All 58 (JSON)