mlsanityData to 14 Oct 2026

Can Local Learning Match Self-Supervised Backpropagation?

arXiv 2601.21683 ↗First posted 29 Jan 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While end-to-end self-supervised learning with backpropagation (global BP-SSL) has become central for training modern AI systems, theories of local self-supervised learning (local-SSL) have struggled to build functional representations in deep neural networks. To establish a link between global and local rules, we first develop a theory for deep linear networks: we identify conditions for local-SSL algorithms (like Forward-forward or CLAPP) to implement exactly the same weight update as a global BP-SSL. Starting from the theoretical insights, we then develop novel variants of local-SSL algorithms to approximate global BP-SSL in deep non-linear convolutional neural networks. Variants that improve the similarity between gradient updates of local-SSL with those of global BP-SSL also show better performance on image datasets (CIFAR-10, STL-10, and Tiny ImageNet). The best local-SSL rule with the CLAPP loss function matches the performance of a comparable global BP-SSL with InfoNCE or CPC-like loss functions, and improves upon state-of-the-art for local SSL on these benchmarks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 89 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

38 of 89 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  7. 2022The Forward-Forward Algorithm: Some Preliminary Investigationscited by 266
  8. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
  9. 2016Direct Feedback Alignment Provides Learning in Deep Neural Networkscited by 240
  10. 2014Difference Target Propagationcited by 148
  11. 2018Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architecturescited by 101
  12. 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 97
26 more
  1. 2019Deep Learning without Weight Transportcited by 87
  2. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 86
  3. 2018Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rulescited by 67
  4. 2020Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Biascited by 64
  5. 2022Hebbian Deep Learning Without Feedbackcited by 61
  6. 2020Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architecturescited by 56
  7. 2020Local plasticity rules can learn deep representations using self-supervised contrastive predictionscited by 47
  8. 2023Backpropagation-free Training of Deep Physical Neural Networkscited by 42
  9. 2023Energy-based learning algorithms for analog computing: a comparative studycited by 37
  10. 2022Towards Scaling Difference Target Propagation by Learning Backprop Targetscited by 34
  11. 2019Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Inputcited by 33
  12. 2024Benchmarking Predictive Coding Networks -- Made Simplecited by 30
  13. 2023Convolutional Channel-wise Competitive Learning for the Forward-Forward Algorithmcited by 25
  14. 2022A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networkscited by 22
  15. 2023Blockwise Self-Supervised Learning at Scalecited by 22
  16. 2020Two Routes to Scalable Credit Assignment without Weight Symmetrycited by 19
  17. 2024Self-Contrastive Forward-Forward Algorithmcited by 17
  18. 2023Forward Learning with Top-Down Feedback: Empirical and Analytical Characterizationcited by 13
  19. 2022Minimizing Control for Credit Assignment with Strong Feedbackcited by 10
  20. 2022Learning efficient backprojections across cortical hierarchies in real timecited by 8
  21. 2023Block-local learning with probabilistic latent representationscited by 6
  22. 2022Scaling Laws Beyond Backpropagationcited by 4
  23. 2025Stochastic Forward-Forward Learning through Representational Dimensionality Compressioncited by 4
  24. 2025ElfCore: A 28nm Neural Processor Enabling Dynamic Structured Sparse Training and Online Self-Supervised Learning with Activity-Dependent Weight Updatecited by 3
  25. 2024Single Neuromorphic Memristor closely Emulates Multiple Synaptic Mechanisms for Energy Efficient Neural Networkscited by 2
  26. 2024EchoSpike Predictive Plasticity: An Online Local Learning Rule for Spiking Neural Networkscited by 2

Built on by

Not cited yet.