# Contrastive Hebbian Learning with Random Feedback Weights

arXiv: https://arxiv.org/abs/1806.07406 · HTML: https://mlsanity.com/paper/1806.07406/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2018.

## Built on by (most cited first, 8 of 8)

- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [SPIDE: A Purely Spike-based Method for Training Feedback Spiking Neural Networks](https://mlsanity.com/paper/2302.00232/) (2023)
- [Spiking Neural Networks and Bio-Inspired Supervised Deep Learning: A Survey](https://mlsanity.com/paper/2307.16235/) (2023)
- [Efficient and Flexible Neural Network Training through Layer-wise Feedback Propagation](https://mlsanity.com/paper/2308.12053/) (2023)
- [Scaling Equilibrium Propagation to Deeper Neural Network Architectures](https://mlsanity.com/paper/2509.26003/) (2025)
- [Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations](https://mlsanity.com/paper/2603.03234/) (2026)
- [aSTDP: A More Biologically Plausible Learning](https://mlsanity.com/paper/2206.14137/) (2022)
- [Minimizing the Effect of Sleep Deprivation in the Forward-Forward Algorithm](https://mlsanity.com/paper/2609.06042/) (2026)

## Abstract (arXiv)

Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are fed to the network, and a backward (or clamped) phase, where the target signals are clamped to the output layer of the network and the feedback signals are transformed through the transpose synaptic weight matrices. This implies symmetries at the synaptic level, for which there is no evidence in the brain. In this work, we propose a new variant of the algorithm, called random contrastive Hebbian learning, which does not rely on any synaptic weights symmetries. Instead, it uses random matrices to transform the feedback signals during the clamped phase, and the neural dynamics are described by first order non-linear differential equations. The algorithm is experimentally verified by solving a Boolean logic task, classification tasks (handwritten digits and letters), and an autoencoding task. This article also shows how the parameters affect learning, especially the random matrices. We use the pseudospectra analysis to investigate further how random matrices impact the learning process. Finally, we discuss the biological plausibility of the proposed algorithm, and how it can give rise to better computational models for learning.