# Training Convolutional Neural Networks with the Forward-Forward algorithm arXiv: https://arxiv.org/abs/2312.14924 ยท HTML: https://mlsanity.com/paper/2312.14924/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 27 of its 49 references that are in the map. - First posted on arXiv 22 Dec 2023. ## Builds on (most cited first, 27 of 27) - [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016) - [Fine-Tuning Language Models with Just Forward Passes](https://mlsanity.com/paper/2305.17333/) (2023) - [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022) - [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016) - [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016) - [Gradients without Backpropagation](https://mlsanity.com/paper/2202.08587/) (2022) - [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020) - [Error-driven Input Modulation: Solving the Credit Assignment Problem without a Backward Pass](https://mlsanity.com/paper/2201.11665/) (2022) - [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022) - [The Predictive Forward-Forward Algorithm](https://mlsanity.com/paper/2301.01452/) (2023) - [SymBa: Symmetric Backpropagation-Free Contrastive Learning with Forward-Forward Algorithm for Optimizing Convergence](https://mlsanity.com/paper/2303.08418/) (2023) - [A Hebbian/Anti-Hebbian Neural Network for Linear Subspace Learning: A Derivation from Multidimensional Scaling of Streaming Data](https://mlsanity.com/paper/1503.00669/) (2015) - [Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm](https://mlsanity.com/paper/2312.12668/) (2023) - [Layer Collaboration in the Forward-Forward Algorithm](https://mlsanity.com/paper/2305.12393/) (2023) - [Forward-Forward Training of an Optical Neural Network](https://mlsanity.com/paper/2305.19170/) (2023) - [The Cascaded Forward Algorithm for Neural Network Training](https://mlsanity.com/paper/2303.09728/) (2023) - [Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks](https://mlsanity.com/paper/2308.09858/) (2023) - [The Trifecta: Three simple techniques for training deeper Forward-Forward networks](https://mlsanity.com/paper/2311.18130/) (2023) - [Emergent representations in networks trained with the Forward-Forward algorithm](https://mlsanity.com/paper/2305.18353/) (2023) - [Graph Neural Networks Go Forward-Forward](https://mlsanity.com/paper/2302.05282/) (2023) - [Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation](https://mlsanity.com/paper/2303.03986/) (2023) - [A theory for the sparsity emerged in the Forward Forward algorithm](https://mlsanity.com/paper/2311.05667/) (2023) - [Activation Learning by Local Competitions](https://mlsanity.com/paper/2209.13400/) (2022) - [Extending the Forward Forward Algorithm](https://mlsanity.com/paper/2307.04205/) (2023) - [Forward-Forward Algorithm for Hyperspectral Image Classification: A Preliminary Study](https://mlsanity.com/paper/2307.00231/) (2023) - [A Study of Forward-Forward Algorithm for Self-Supervised Learning](https://mlsanity.com/paper/2309.11955/) (2023) - [The Forward-Forward Algorithm as a feature extractor for skin lesion classification: A preliminary study](https://mlsanity.com/paper/2307.00617/) (2023) ## Built on by (most cited first, 14 of 14) - [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024) - [Contrastive Forward-Forward: A Training Algorithm of Vision Transformer](https://mlsanity.com/paper/2502.00571/) (2025) - [Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation](https://mlsanity.com/paper/2509.12394/) (2025) - [Hyperspherical Forward-Forward with Prototypical Representations](https://mlsanity.com/paper/2605.00082/) (2026) - [A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks](https://mlsanity.com/paper/2409.07387/) (2024) - [Scalable Forward-Forward Algorithm](https://mlsanity.com/paper/2501.03176/) (2025) - [Function regression using the forward forward training and inferring paradigm](https://mlsanity.com/paper/2510.06762/) (2025) - [HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm](https://mlsanity.com/paper/2605.24797/) (2026) - [On Advancements of the Forward-Forward Algorithm](https://mlsanity.com/paper/2504.21662/) (2025) - [Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective](https://mlsanity.com/paper/2509.08697/) (2025) - [Local learning for stable backpropagation-free neural network training towards physical learning](https://mlsanity.com/paper/2603.24790/) (2026) - [Forward-Cooperation-Backward (FCB) learning in a Multi-Encoding Uni-Decoding neural network architecture](https://mlsanity.com/paper/2502.20113/) (2025) - [The Price of Locality: Why Forward-Forward Underperforms Backpropagation?](https://mlsanity.com/paper/2609.33240/) (2026) - [Rescuing Points Trapped Inside Incorrect Decision Regions: A target for Evaluating and Improving Forward-Forward Learning](https://mlsanity.com/paper/2610.16347/) (2026) ## Abstract (arXiv) Recent successes in image analysis with deep neural networks are achieved almost exclusively with Convolutional Neural Networks (CNNs), typically trained using the backpropagation (BP) algorithm. In a 2022 preprint, Geoffrey Hinton proposed the Forward-Forward (FF) algorithm as a biologically inspired alternative, where positive and negative examples are jointly presented to the network and training is guided by a locally defined goodness function. Here, we extend the FF paradigm to CNNs. We introduce two spatially extended labeling strategies, based on Fourier patterns and morphological transformations, that enable convolutional layers to access label information across all spatial positions. On CIFAR10, we show that deeper FF-trained CNNs can be optimized successfully and that morphology-based labels prevent shortcut solutions on dataset with more complex and fine features. On CIFAR100, carefully designed label sets scale effectively to 100 classes. Class Activation Maps reveal that FF-trained CNNs learn meaningful and complementary features across layers. Together, these results demonstrate that FF training is feasible beyond fully connected networks, provide new insights into its learning dynamics and stability, and highlight its potential for neuromorphic computing and biologically inspired learning.