# Efficiency of local learning rules in threshold-linear associative networks

arXiv: https://arxiv.org/abs/2007.12584 · HTML: https://mlsanity.com/paper/2007.12584/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2020.

## Built on by (most cited first, 6 of 6)

- [Restricted Boltzmann Machines as Models of Interacting Variables](https://mlsanity.com/paper/2103.15917/) (2021)
- [Linear Classification of Neural Manifolds with Correlated Variability](https://mlsanity.com/paper/2211.14961/) (2022)
- [Memorization and Knowledge Injection in Gated LLMs](https://mlsanity.com/paper/2504.21239/) (2025)
- [Pattern reconstruction with restricted Boltzmann machines](https://mlsanity.com/paper/2205.07087/) (2022)
- [Dynamical stability for dense patterns in attractor neural networks](https://mlsanity.com/paper/2507.10383/) (2025)
- [Activation Functions, Statistics and Learning of Higher-Order Interactions in Restricted Boltzmann Machines](https://mlsanity.com/paper/2605.19178/) (2026)

## Abstract (arXiv)

We derive the Gardner storage capacity for associative networks of threshold linear units, and show that with Hebbian learning they can operate closer to such Gardner bound than binary networks, and even surpass it. This is largely achieved through a sparsification of the retrieved patterns, which we analyze for theoretical and empirical distributions of activity. As reaching the optimal capacity via non-local learning rules like backpropagation requires slow and neurally implausible training procedures, our results indicate that one-shot self-organized Hebbian learning can be just as efficient.