# Scalable Forward-Forward Algorithm

arXiv: https://arxiv.org/abs/2501.03176 · HTML: https://mlsanity.com/paper/2501.03176/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm](https://mlsanity.com/paper/2312.12668/) (2023)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Training Convolutional Neural Networks with the Forward-Forward algorithm](https://mlsanity.com/paper/2312.14924/) (2023)
- [Improved Forward-Forward Contrastive Learning](https://mlsanity.com/paper/2405.03432/) (2024)
- [Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networks](https://mlsanity.com/paper/2406.14038/) (2024)

## Built on by (most cited first, 2 of 2)

- [Hyperspherical Forward-Forward with Prototypical Representations](https://mlsanity.com/paper/2605.00082/) (2026)
- [Cumulative-Goodness Free-Riding in Forward-Forward Networks: Real, Repairable, but Not Accuracy-Dominant](https://mlsanity.com/paper/2605.06240/) (2026)

## Abstract (arXiv)

We propose a scalable Forward-Forward (FF) algorithm that eliminates the need for backpropagation by training each layer separately. Unlike backpropagation, FF avoids backward gradients and can be more modular and memory efficient, making it appealing for large networks. We extend FF to modern convolutional architectures, such as MobileNetV3 and ResNet18, by introducing a new way to compute losses for convolutional layers. Experiments show that our method achieves performance comparable to standard backpropagation. Furthermore, when we divide the network into blocks, such as the residual blocks in ResNet, and apply backpropagation only within each block, but not across blocks, our hybrid design tends to outperform backpropagation baselines while maintaining a similar training speed. Finally, we present experiments on small datasets and transfer learning that confirm the adaptability of our method.