mlsanityData to 14 Oct 2026

The Predictive Forward-Forward Algorithm

arXiv 2301.01452 ↗First posted 4 Jan 2023cs.LG cs.NEcited by 35

Not on the map. Why?

Abstract (arXiv)

We propose the predictive forward-forward (PFF) algorithm for conducting credit assignment in neural systems. Specifically, we design a novel, dynamic recurrent neural system that learns a directed generative circuit jointly and simultaneously with a representation circuit. Notably, the system integrates learnable lateral competition, noise injection, and elements of predictive coding, an emerging and viable neurobiological process theory of cortical function, with the forward-forward (FF) adaptation scheme. Furthermore, PFF efficiently learns to propagate learning signals and updates synapses with forward passes only, eliminating key structural and computational constraints imposed by backpropagation-based schemes. Besides computational advantages, the PFF process could prove useful for understanding the learning mechanisms behind biological neurons that use local signals despite missing feedback connections. We run experiments on image data and demonstrate that the PFF procedure works as well as backpropagation, offering a promising brain-inspired algorithm for classifying, reconstructing, and synthesizing data patterns.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 29 of its 62 references that are in the map. First posted on arXiv 4 Jan 2023.

Builds on

29 of 62 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
  4. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
  5. 2018Deep Learning for Classical Japanese Literaturecited by 422
  6. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 266
  7. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
  8. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  9. 2014Difference Target Propagationcited by 148
  10. 2016Decoupled Neural Interfaces using Synthetic Gradientscited by 137
  11. 2019Training Neural Networks with Local Error Signalscited by 136
  12. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 101
17 more
  1. 2016Towards an integration of deep learning and neurosciencecited by 78
  2. 2019Decoupled Greedy Learning of CNNscited by 78
  3. 2014Reweighted Wake-Sleepcited by 66
  4. 2022Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Passcited by 63
  5. 2014Random feedback weights support learning in deep neural networkscited by 47
  6. 2020The Neural Coding Framework for Learning Generative Modelscited by 44
  7. 2021Associative Memories via Predictive Codingcited by 44
  8. 2018Biologically Motivated Algorithms for Propagating Local Target Representationscited by 43
  9. 2013Deep Generative Stochastic Networks Trainable by Backpropcited by 36
  10. 2022Signal Propagation: A Framework for Learning and Inference In a Forward Passcited by 24
  11. 2021Reverse Differentiation via Predictive Codingcited by 23
  12. 2019Spiking Neural Predictive Coding for Continual Learning from Data Streamscited by 17
  13. 2018Error Forward-Propagation: Reusing Feedforward Connections to Propagate Errors in Deep Learningcited by 16
  14. 2017Learning to Adapt by Minimizing Discrepancycited by 12
  15. 2014Kickback cuts Backprop's red-tape: Biologically plausible credit assignment in neural networkscited by 11
  16. 2020Afro-MNIST: Synthetic generation of MNIST-style datasets for low-resource languagescited by 5
  17. 2022An Empirical Analysis of Recurrent Learning Algorithms In Neural Lossy Image Compression Systemscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 11; 2025: 7; 2026: 5.

35 papers.

Sort:

By area:

  1. 2023Brain-inspired Computational Intelligence via Predictive Codingcited by 30
  2. 2023The Cascaded Forward Algorithm for Neural Network Trainingcited by 17
  3. 2024Self-Contrastive Forward-Forward Algorithmcited by 17
  4. 2023Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignmentcited by 15
  5. 2023Training Convolutional Neural Networks with the Forward-Forward algorithmcited by 14
  6. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
  7. 2023The Trifecta: Three simple techniques for training deeper Forward-Forward networkscited by 12
  8. 2023Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuitscited by 10
  9. 2023Block-local learning with probabilistic latent representationscited by 6
  10. 2023Extending the Forward Forward Algorithmcited by 6
  11. 2024A Review of Neuroscience-Inspired Machine Learningcited by 6
  12. 2025Backpropagation-free Spiking Neural Networks with the Forward-Forward Algorithmcited by 5
23 more
  1. 2023Flexible Phase Dynamics for Bio-Plausible Contrastive Learningcited by 4
  2. 2023A Study of Forward-Forward Algorithm for Self-Supervised Learningcited by 4
  3. 2025Mono-Forward: Revisiting Forward-Forward through Objective-Locality Decompositioncited by 4
  4. 2023Feed-Forward Optimization With Delayed Feedback for Neural Network Trainingcited by 3
  5. 2024LightFF: Lightweight Inference for Forward-Forward Algorithmcited by 3
  6. 2024Employing Layerwised Unsupervised Learning to Lessen Data and Loss Requirements in Forward-Forward Algorithmscited by 2
  7. 2024Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networkscited by 2
  8. 2024A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Taskscited by 2
  9. 2026HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithmcited by 2
  10. 2024Forward Learning of Graph Neural Networkscited by 1
  11. 2024Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computingcited by 1
  12. 2024On the Improvement of Generalization and Stability of Forward-Only Learning via Neural Polarizationcited by 1
  13. 2024A Unified Framework for Neural Computation and Learning Over Timecited by 1
  14. 2024Nature-Inspired Local Propagationcited by 0
  15. 2025Forward-Cooperation-Backward (FCB) learning in a Multi-Encoding Uni-Decoding neural network architecturecited by 0
  16. 2025Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learningcited by 0
  17. 2025Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Lossescited by 0
  18. 2025Local Pairwise Distance Matching for Backpropagation-Free Reinforcement Learningcited by 0
  19. 2025Local Reinforcement Learning with Action-Conditioned Root Mean Squared Q-Functionscited by 0
  20. 2026Selectivity and Shape in the Design of Forward-Forward Goodness Functionscited by 0
  21. 2026Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Trainingcited by 0
  22. 2026Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignmentcited by 0
  23. 2026Gauge-Fixing the Forward-Forward Objective: A Whitened Goodness Derived from a Likelihood-Ratio Accountcited by 0