mlsanityData to 7 Oct 2026

A theory for the sparsity emerged in the Forward Forward algorithm

arXiv 2311.05667 ↗First posted 9 Nov 2023cs.LG cs.AIcited by 7

Not on the map. Why?

Abstract (arXiv)

This report explores the theory that explains the high sparsity phenomenon citeptosato2023emergent observed in the forward-forward algorithm citephinton2022forward. The two theorems proposed predict the sparsity changes of a single data point's activation in two cases: Theorem reftheorem:1: Decrease the goodness of the whole batch. Theorem reftheorem:2: Apply the complete forward forward algorithm to decrease the goodness for negative data and increase the goodness for positive data. The theory aligns well with the experiments tested on the MNIST dataset.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 3 of its 5 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

3 of 5 references.

References · 3

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  2. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 262
  3. 2023Emergent representations in networks trained with the Forward-Forward algorithmcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 0; 2026: 1.

7 papers.

  1. 2024Self-Contrastive Forward-Forward Algorithmcited by 15
  2. 2023Training Convolutional Neural Networks with the Forward-Forward algorithmcited by 13
  3. 2024Resource-efficient Medical Image Analysis with Self-adapting Forward-Forward Networkscited by 2
  4. 2024A Contrastive Symmetric Forward-Forward Algorithm (SFFA) for Continual Learning Taskscited by 2
  5. 2024Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computingcited by 1
  6. 2024On the Improvement of Generalization and Stability of Forward-Only Learning via Neural Polarizationcited by 1
  7. 2026Forward-Only Convolutional Neural Networks with Learnable Channel-Class Assignmentcited by 0