# The Predictive Forward-Forward Algorithm

arXiv: https://arxiv.org/abs/2301.01452 · HTML: https://mlsanity.com/paper/2301.01452/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 62 references that are in the map.
- First posted on arXiv 4 Jan 2023.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Deep Learning for Classical Japanese Literature](https://mlsanity.com/paper/1812.01718/) (2018)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Difference Target Propagation](https://mlsanity.com/paper/1412.7525/) (2014)
- [Decoupled Neural Interfaces using Synthetic Gradients](https://mlsanity.com/paper/1608.05343/) (2016)
- [Training Neural Networks with Local Error Signals](https://mlsanity.com/paper/1901.06656/) (2019)
- [Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures](https://mlsanity.com/paper/1807.04587/) (2018)
- [Towards an integration of deep learning and neuroscience](https://mlsanity.com/paper/1606.03813/) (2016)
- [Decoupled Greedy Learning of CNNs](https://mlsanity.com/paper/1901.08164/) (2019)
- [Reweighted Wake-Sleep](https://mlsanity.com/paper/1406.2751/) (2014)
- [Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass](https://mlsanity.com/paper/2201.11665/) (2022)
- [Random feedback weights support learning in deep neural networks](https://mlsanity.com/paper/1411.0247/) (2014)
- [The Neural Coding Framework for Learning Generative Models](https://mlsanity.com/paper/2012.03405/) (2020)
- [Associative Memories via Predictive Coding](https://mlsanity.com/paper/2109.08063/) (2021)
- [Biologically Motivated Algorithms for Propagating Local Target Representations](https://mlsanity.com/paper/1805.11703/) (2018)
- [Deep Generative Stochastic Networks Trainable by Backprop](https://mlsanity.com/paper/1306.1091/) (2013)
- [Signal Propagation: A Framework for Learning and Inference In a Forward Pass](https://mlsanity.com/paper/2204.01723/) (2022)
- [Reverse Differentiation via Predictive Coding](https://mlsanity.com/paper/2103.04689/) (2021)
- [Spiking Neural Predictive Coding for Continual Learning from Data Streams](https://mlsanity.com/paper/1908.08655/) (2019)
- [Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learning](https://mlsanity.com/paper/1808.03357/) (2018)
- [Learning to Adapt by Minimizing Discrepancy](https://mlsanity.com/paper/1711.11542/) (2017)
- [Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networks](https://mlsanity.com/paper/1411.6191/) (2014)
- [Afro-MNIST: Synthetic generation of MNIST-style datasets for low-resource languages](https://mlsanity.com/paper/2009.13509/) (2020)
- [An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systems](https://mlsanity.com/paper/2201.11782/) (2022)

## Built on by (most cited first, 30 of 35)

- [Brain-inspired Computational Intelligence via Predictive Coding](https://mlsanity.com/paper/2308.07870/) (2023)
- [The Cascaded Forward Algorithm for Neural Network Training](https://mlsanity.com/paper/2303.09728/) (2023)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Training Convolutional Neural Networks with the Forward-Forward algorithm](https://mlsanity.com/paper/2312.14924/) (2023)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [The Trifecta: Three simple techniques for training deeper Forward-Forward networks](https://mlsanity.com/paper/2311.18130/) (2023)
- [Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits](https://mlsanity.com/paper/2303.18187/) (2023)
- [Block-local learning with probabilistic latent representations](https://mlsanity.com/paper/2305.14974/) (2023)
- [Extending the Forward Forward Algorithm](https://mlsanity.com/paper/2307.04205/) (2023)
- [A Review of Neuroscience-Inspired Machine Learning](https://mlsanity.com/paper/2403.18929/) (2024)
- [Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithm](https://mlsanity.com/paper/2502.20411/) (2025)
- [Flexible Phase Dynamics for Bio-Plausible Contrastive Learning](https://mlsanity.com/paper/2302.12431/) (2023)
- [A Study of Forward-Forward Algorithm for Self-Supervised Learning](https://mlsanity.com/paper/2309.11955/) (2023)
- [Mono-Forward: Revisiting Forward-Forward through Objective-Locality Decomposition](https://mlsanity.com/paper/2501.09238/) (2025)
- [Feed-Forward Optimization With Delayed Feedback for Neural Network Training](https://mlsanity.com/paper/2304.13372/) (2023)
- [LightFF: Lightweight Inference for Forward-Forward Algorithm](https://mlsanity.com/paper/2404.05241/) (2024)
- [Employing Layerwised Unsupervised Learning to Lessen Data and Loss Requirements in Forward-Forward Algorithms](https://mlsanity.com/paper/2404.14664/) (2024)
- [Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networks](https://mlsanity.com/paper/2406.14038/) (2024)
- [A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks](https://mlsanity.com/paper/2409.07387/) (2024)
- [HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm](https://mlsanity.com/paper/2605.24797/) (2026)
- [Forward Learning of Graph Neural Networks](https://mlsanity.com/paper/2403.11004/) (2024)
- [Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computing](https://mlsanity.com/paper/2406.16479/) (2024)
- [On the Improvement of Generalization and Stability of Forward-Only Learning via Neural Polarization](https://mlsanity.com/paper/2408.09210/) (2024)
- [A Unified Framework for Neural Computation and Learning Over Time](https://mlsanity.com/paper/2409.12038/) (2024)
- [Nature-Inspired Local Propagation](https://mlsanity.com/paper/2402.05959/) (2024)
- [Forward-Cooperation-Backward (FCB) learning in a Multi-Encoding Uni-Decoding neural network architecture](https://mlsanity.com/paper/2502.20113/) (2025)
- [Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learning](https://mlsanity.com/paper/2503.21796/) (2025)
- [Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses](https://mlsanity.com/paper/2506.11030/) (2025)
- [Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learning](https://mlsanity.com/paper/2507.11367/) (2025)

## Abstract (arXiv)

We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems. Specifically, we design a novel, dynamic recurrent neural system that learns a directed generative circuit jointly and simultaneously with a representation circuit. Notably, the system integrates learnable lateral competition, noise injection, and elements of predictive coding, an emerging and viable neurobiological process theory of cortical function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF efficiently learns to propagate learning signals and updates synapses with forward passes only, eliminating key structural and computational constraints imposed by backpropagation-based schemes. Besides computational advantages, the PFF process could prove useful for understanding the learning mechanisms behind biological neurons that use local signals despite missing feedback connections. We run experiments on image data and demonstrate that the PFF procedure works as well as backpropagation, offering a promising brain-inspired algorithm for classifying, reconstructing, and synthesizing data patterns.