mlsanityData to 14 Oct 2026

Gauge-Fixing the Forward-Forward Objective: A Whitened Goodness Derived from a Likelihood-Ratio Account

arXiv 2607.12501 ↗First posted 14 Jul 2026cs.LG eess.IV stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

The Forward-Forward algorithm trains each layer locally, so that a scalar goodness - the sum of squared activations - is high on real inputs and low on contrastive ones. Under an explicit generative model this goodness is the sufficient statistic of a likelihood-ratio test, and the pairwise form of the objective admits a gauge: a layer can lower its loss by inflating the scale of its weights rather than by separating the two populations. The analysis prescribes the repair - a whitened, scale-invariant goodness trained online within each layer - which we evaluate as a training procedure. Across three corpora, three depths and a fourfold range of layer width (13 seeds per cell), it raises linear-probe accuracy over the standard pairwise objective in every measured cell - by 4 to 7 points on eight of nine corpus-depth combinations - and closes 16-61% of the gap to end-to-end backpropagation. A control isolates the mechanism: Hinton's fixed-threshold loss also bounds the runaway, to a factor of 1.4 against 133, yet tracks the unmodified baseline - invariance to the gauge, not a bound on it, is what pays. Against the strongest published alternative - a sparse, top-k goodness - the derived objective is statistically indistinguishable on two corpora of three, yet only it removes the runaway: sparsity and gauge-invariance are independent axes, and the published variant recovers accuracy while leaving the pathology in place. We state the boundaries we measured, and every prediction was recorded before its experiment with the refutations reported.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 14 Jul 2026.

Builds on

13 of 28 references.

References · 13

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 266
  4. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  5. 2019Training Neural Networks with Local Error Signalscited by 136
  6. 2018Greedy Layerwise Learning Can Scale to ImageNetcited by 115
  7. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 86
  8. 2023The Predictive Forward-Forward Algorithmcited by 35
  9. 2023Layer Collaboration in the Forward-Forward Algorithmcited by 21
  10. 2023The Cascaded Forward Algorithm for Neural Network Trainingcited by 17
  11. 2025In Search of Goodness: Large Scale Benchmarking of Goodness Functions for the Forward-Forward Algorithmcited by 2
  12. 2026Covariance-Aware Goodness for Scalable Forward-Forward Learningcited by 2
1 more
  1. 2026Partial-Reference IQA Based on Hermite-Gauss Structural Prediction and Texture Deviationcited by 1

Built on by

Not cited yet.