# Layer Collaboration in the Forward-Forward Algorithm

arXiv: https://arxiv.org/abs/2305.12393 · HTML: https://mlsanity.com/paper/2305.12393/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies](https://mlsanity.com/paper/2010.10802/) (2020)
- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Perceptual Score: What Data Modalities Does Your Model Perceive?](https://mlsanity.com/paper/2110.14375/) (2021)
- [Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions](https://mlsanity.com/paper/2106.04484/) (2021)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [Bottom-up Broadcast Neural Network For Music Genre Classification](https://mlsanity.com/paper/1901.08928/) (2019)
- [Syntactic and Semantic Features For Code-Switching Factored Language Models](https://mlsanity.com/paper/1710.01809/) (2017)
- [Learning Randomly Perturbed Structured Predictors for Direct Loss Minimization](https://mlsanity.com/paper/2007.05724/) (2020)
- [On the Importance of Gradient Norm in PAC-Bayesian Bounds](https://mlsanity.com/paper/2210.06143/) (2022)
- [Learning Deep Energy Models: Contrastive Divergence vs. Amortized MLE](https://mlsanity.com/paper/1707.00797/) (2017)
- [A Functional Information Perspective on Model Interpretation](https://mlsanity.com/paper/2206.05700/) (2022)

## Built on by (most cited first, 21 of 21)

- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Training Convolutional Neural Networks with the Forward-Forward algorithm](https://mlsanity.com/paper/2312.14924/) (2023)
- [Contrastive Forward-Forward: A Training Algorithm of Vision Transformer](https://mlsanity.com/paper/2502.00571/) (2025)
- [Mono-Forward: Revisiting Forward-Forward through Objective-Locality Decomposition](https://mlsanity.com/paper/2501.09238/) (2025)
- [Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation](https://mlsanity.com/paper/2509.12394/) (2025)
- [Hyperspherical Forward-Forward with Prototypical Representations](https://mlsanity.com/paper/2605.00082/) (2026)
- [Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networks](https://mlsanity.com/paper/2406.14038/) (2024)
- [A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Tasks](https://mlsanity.com/paper/2409.07387/) (2024)
- [Function regression using the forward forward training and inferring paradigm](https://mlsanity.com/paper/2510.06762/) (2025)
- [HCL-FF: Hierarchical and Contrastive Learning for Forward-Forward Algorithm](https://mlsanity.com/paper/2605.24797/) (2026)
- [ERASE: EaRly bAckpropagation SchEdule for Faster Training of Modern Recommendation Systems](https://mlsanity.com/paper/2608.18469/) (2026)
- [Forward Learning of Graph Neural Networks](https://mlsanity.com/paper/2403.11004/) (2024)
- [Stochastic Layer-wise Learning: Scalable and Efficient Alternative to Backpropagation](https://mlsanity.com/paper/2505.05181/) (2025)
- [Reshaping the Forward-Forward Algorithm with a Similarity-Based Objective](https://mlsanity.com/paper/2509.08697/) (2025)
- [Forward-Cooperation-Backward (FCB) learning in a Multi-Encoding Uni-Decoding neural network architecture](https://mlsanity.com/paper/2502.20113/) (2025)
- [Selectivity and Shape in the Design of Forward-Forward Goodness Functions](https://mlsanity.com/paper/2604.13081/) (2026)
- [Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant](https://mlsanity.com/paper/2605.06240/) (2026)
- [FFR: Forward-Forward Learning for Regression](https://mlsanity.com/paper/2606.03927/) (2026)
- [Gauge-Fixing the Forward-Forward Objective: A Whitened Goodness Derived from a Likelihood-Ratio Account](https://mlsanity.com/paper/2607.12501/) (2026)
- [Emergent Hierarchical Monosemantic Neurons from the Group-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2607.16295/) (2026)
- [Rescuing Points Trapped Inside Incorrect Decision Regions: A target for Evaluating and Improving Forward-Forward Learning](https://mlsanity.com/paper/2610.16347/) (2026)

## Abstract (arXiv)

Backpropagation, which uses the chain rule, is the de-facto standard algorithm for optimizing neural networks nowadays. Recently, Hinton (2022) proposed the forward-forward algorithm, a promising alternative that optimizes neural nets layer-by-layer, without propagating gradients throughout the network. Although such an approach has several advantages over back-propagation and shows promising results, the fact that each layer is being trained independently limits the optimization process. Specifically, it prevents the network's layers from collaborating to learn complex and rich features. In this work, we study layer collaboration in the forward-forward algorithm. We show that the current version of the forward-forward algorithm is suboptimal when considering information flow in the network, resulting in a lack of collaboration between layers of the network. We propose an improved version that supports layer collaboration to better utilize the network structure, while not requiring any additional assumptions or computations. We empirically demonstrate the efficacy of the proposed version when considering both information flow and objective metrics. Additionally, we provide a theoretical motivation for the proposed method, inspired by functional entropy theory.