# Stochastic Forward-Forward Learning through Representational Dimensionality Compression

arXiv: https://arxiv.org/abs/2505.16649 · HTML: https://mlsanity.com/paper/2505.16649/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 56 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 18 of 18)

- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Putting An End to End-to-End: Gradient-Isolated Learning of Representations](https://mlsanity.com/paper/1905.11786/) (2019)
- [Hebbian Deep Learning Without Feedback](https://mlsanity.com/paper/2209.11883/) (2022)
- [Local plasticity rules can learn deep representations using self-supervised contrastive predictions](https://mlsanity.com/paper/2010.08262/) (2020)
- [Dimensionality compression and expansion in Deep Neural Networks](https://mlsanity.com/paper/1906.00443/) (2019)
- [Energy-based learning algorithms for analog computing: a comparative study](https://mlsanity.com/paper/2312.15103/) (2023)
- [Neural tuning and representational geometry](https://mlsanity.com/paper/2104.09743/) (2021)
- [Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithm](https://mlsanity.com/paper/2312.12668/) (2023)
- [The Cascaded Forward Algorithm for Neural Network Training](https://mlsanity.com/paper/2303.09728/) (2023)
- [Bits and Pieces: Understanding Information Decomposition from Part-whole Relationships and Formal Logic](https://mlsanity.com/paper/2008.09535/) (2020)
- [Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment](https://mlsanity.com/paper/2312.09257/) (2023)
- [Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks](https://mlsanity.com/paper/2501.17266/) (2025)
- [Toward stochastic neural computing](https://mlsanity.com/paper/2305.13982/) (2023)
- [Towards free-response paradigm: a theory on decision-making in spiking neural networks](https://mlsanity.com/paper/2404.10599/) (2024)

## Built on by (most cited first, 4 of 4)

- [AutoSOTA: An End-to-End Automated Research System for State-of-the-Art AI Model Discovery](https://mlsanity.com/paper/2604.05550/) (2026)
- [Discriminative Flow Matching Via Local Generative Predictors](https://mlsanity.com/paper/2603.13928/) (2026)
- [Can Local Learning Match Self-Supervised Backpropagation?](https://mlsanity.com/paper/2601.21683/) (2026)
- [FFR: Forward-Forward Learning for Regression](https://mlsanity.com/paper/2606.03927/) (2026)

## Abstract (arXiv)

The Forward-Forward (FF) learning algorithm provides a bottom-up alternative to backpropagation (BP) for training neural networks, relying on a layer-wise "goodness" function with well-designed negative samples for contrastive learning. Existing goodness functions are typically defined as the sum of squared postsynaptic activations, neglecting correlated variability between neurons. In this work, we propose a novel goodness function termed dimensionality compression that uses the effective dimensionality (ED) of fluctuating neural responses to incorporate second-order statistical structure. Our objective minimizes ED for noisy copies of individual inputs while maximizing it across the sample distribution, promoting structured representations without the need to prepare negative samples.We demonstrate that this formulation achieves competitive performance compared to other non-BP methods. Moreover, we show that noise plays a constructive role that can enhance generalization and improve inference when predictions are derived from the mean of squared output, which is equivalent to making predictions based on an energy term. Our findings contribute to the development of more biologically plausible learning algorithms and suggest a natural fit for neuromorphic computing, where stochasticity is a computational resource rather than a nuisance. The code is available at https://github.com/ZhichaoZhu/StochasticForwardForward