mlsanityData to 7 Oct 2026

Unlocking the Potential of Similarity Matching: Scalability, Supervision and Pre-training

arXiv 2308.02427 ↗First posted 2 Aug 2023cs.NE cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While effective, the backpropagation (BP) algorithm exhibits limitations in terms of biological plausibility, computational cost, and suitability for online learning. As a result, there has been a growing interest in developing alternative biologically plausible learning approaches that rely on local learning rules. This study focuses on the primarily unsupervised similarity matching (SM) framework, which aligns with observed mechanisms in biological systems and offers online, localized, and biologically plausible algorithms. i) To scale SM to large datasets, we propose an implementation of Convolutional Nonnegative SM using PyTorch. ii) We introduce a localized supervised SM objective reminiscent of canonical correlation analysis, facilitating stacking SM layers. iii) We leverage the PyTorch implementation for pre-training architectures such as LeNet and compare the evaluation of features against BP-trained models. This work combines biologically plausible algorithms with computational efficiency opening multiple avenues for further explorations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 36 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

17 of 36 references.

References · 17

  1. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 722
  2. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 251
  3. 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
  4. 2020On the Theory of Transfer Learning: The Importance of Task Diversitycited by 186
  5. 2019Training Neural Networks with Local Error Signalscited by 135
  6. 2018Greedy Layerwise Learning Can Scale to ImageNetcited by 113
  7. 2018Dendritic cortical microcircuits approximate the backpropagation algorithmcited by 96
  8. 2019The HSIC Bottleneck: Deep Learning without Back-Propagationcited by 77
  9. 2020Kernelized information bottleneck leads to biologically plausible 3-factor Hebbian learning in deep networkscited by 25
  10. 2017Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?cited by 20
  11. 2019Neuroscience-inspired online unsupervised learning algorithmscited by 19
  12. 2022Constrained Predictive Coding as a Biologically Plausible Model of the Cortical Hierarchycited by 17
5 more
  1. 2019Structured and Deep Similarity Matching via Structured and Deep Hebbian Networkscited by 9
  2. 2015A Hebbian/Anti-Hebbian Network for Online Sparse Dictionary Learning Derived from Symmetric Matrix Factorizationcited by 6
  3. 2021Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methodscited by 5
  4. 2013Unsupervised Feature Learning by Deep Sparse Codingcited by 4
  5. 2019A Neural Network for Semi-Supervised Learning on Manifoldscited by 1

Built on by

Not cited yet.