# Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories

arXiv: https://arxiv.org/abs/2307.08365 · HTML: https://mlsanity.com/paper/2307.08365/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 69 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 18 of 18)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Dense Associative Memory for Pattern Recognition](https://mlsanity.com/paper/1606.01164/) (2016)
- [Restricted Boltzmann Machine, recent advances and mean-field theory](https://mlsanity.com/paper/2011.11307/) (2020)
- [Phase Diagram of Restricted Boltzmann Machines and Generalised Hopfield Networks with Arbitrary Priors](https://mlsanity.com/paper/1702.05882/) (2017)
- [Emergence of Compositional Representations in Restricted Boltzmann Machines](https://mlsanity.com/paper/1611.06759/) (2016)
- [Phase transitions in Restricted Boltzmann Machines with generic priors](https://mlsanity.com/paper/1612.03132/) (2016)
- [The emergence of a concept in shallow neural networks](https://mlsanity.com/paper/2109.00454/) (2021)
- [Supervised Hebbian Learning](https://mlsanity.com/paper/2203.01304/) (2022)
- [Dreaming neural networks: forgetting spurious memories and reinforcing pure ones](https://mlsanity.com/paper/1810.12217/) (2018)
- [Storage and Learning phase transitions in the Random-Features Hopfield Model](https://mlsanity.com/paper/2303.16880/) (2023)
- [Dense Hebbian neural networks: a replica symmetric picture of unsupervised learning](https://mlsanity.com/paper/2211.14067/) (2022)
- [Dense Hebbian neural networks: a replica symmetric picture of supervised learning](https://mlsanity.com/paper/2212.00606/) (2022)
- [Hopfield model with planted patterns: a teacher-student self-supervised learning model](https://mlsanity.com/paper/2304.13710/) (2023)
- [Power-law Scaling to Assist with Key Challenges in Artificial Intelligence](https://mlsanity.com/paper/2211.08430/) (2022)
- [Efficient shallow learning as an alternative to deep learning](https://mlsanity.com/paper/2211.11106/) (2022)
- [From Dyson to Hopfield: Processing on hierarchical networks](https://mlsanity.com/paper/1407.5019/) (2014)
- [Thermodynamics of bidirectional associative memories](https://mlsanity.com/paper/2211.09694/) (2022)

## Built on by (most cited first, 1 of 1)

- [Exponential Capacity in Multilayer Hetero-Associative Neural Networks](https://mlsanity.com/paper/2607.29554/) (2026)

## Abstract (arXiv)

We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative -- involving couples of patterns -- and it is achieved by reverberating the information depicted from the examples through the layers of the network. By adapting Guerra's interpolation technique, we provide a full statistical mechanical picture of supervised and unsupervised learning processes (at the replica symmetric level of description) obtaining analytically phase diagrams, thresholds for learning, a picture of the ground-state in plain agreement with Monte Carlo simulations and signal-to-noise outcomes. In the large dataset limit, the Kosko storage prescription as well as its statistical mechanical picture provided by Kurchan, Peliti, and Saber in the eighties is fully recovered. Computational advantages in dealing with information reverberation, rather than storage, are discussed for natural test cases. In particular, we show how this network admits an integral representation in terms of two coupled restricted Boltzmann machines, whose hidden layers are entirely built of by grand-mother neurons, to prove that by coupling solely these grand-mother neurons we can correlate the patterns they are related to: it is thus possible to recover Pavlov's Classical Conditioning by adding just one synapse among the correct grand-mother neurons (hence saving an extensive number of these links for further information storage w.r.t. the classical autoassociative setting).